ABSTRACT Wendy A. Miller, AN EVALUATION OF A SCHOOL SYSTEM'S EFFORT TO DEVELOP AND IMPLEMENT A LEADERSHIP PREPARATION PROGRAM (Under the direction of Dr. James McDowelle) Department of Educational Leadership, March, 2015. This study is a program evaluation of one school system’s effort to develop and implement a leadership preparation program. The study was designed to determine if the implementation of the GROW program addressed the problem of the lack of qualified and interested leadership candidates currently within the district. Data were gathered to determine if the completers of the district-designed GROW program had an increased level of interest in securing school leadership positions. The framework for the study was based on Donald Kirkpatrick's (2006) Model of Program Evaluation and included four levels of evaluation: (1) reaction, (2) learning, (3) behavior, and (4) overall results of the program. Quantitative and qualitative evaluation methods used in the study included surveys, questionnaires, pre and post assessments, and individual interviews. The findings of the study indicate that the GROW program was effective for participants and beneficial to the school district. GROW participants reported a high level of satisfaction with the program and felt better prepared for school leadership positions. The research revealed emergent themes that supported participant growth in key leadership principles. The research also supported the value of the cohort model, the powerful influence of “tapping”, and advantages of a district run program as compared to university programs. Ninety-five percent of GROW program participants reported an increased level of interest in school leadership. The GROW program was perceived as effective in increasing the number of interested and qualified candidates for school leadership positions in the host district. Further, preliminary data from this study was used in a grant proposal to the Z. Smith Reynolds Foundation seeking funding support for leadership initiatives in Craven County Schools. The grant was awarded in the amount of ninety thousand dollars. The results of this study have impacted the future of leadership initiatives in Craven County Schools. AN EVALUATION OF A SCHOOL SYSTEM'S EFFORT TO DEVELOP AND IMPLEMENT A LEADERSHIP PREPARATION PROGRAM A Dissertation Presented to the Faculty of the Department of Educational Leadership East Carolina University In Partial Fulfillment of the Requirements for the Degree Doctor of Education in Educational Leadership by Wendy A. Miller March, 2015 ©Copyright 2015 Wendy A. Miller AN EVALUATION OF A SCHOOL SYSTEM'S EFFORT TO DEVELOP AND IMPLEMENT A LEADERSHIP PREPARATION PROGRAM by Wendy A. Miller APPROVED BY: DIRECTOR OF DISSERTATION:_________________________________________________ James McDowelle, EdD COMMITTEE MEMBER:________________________________________________________ William A. Rouse, Jr., EdD COMMITTEE MEMBER:________________________________________________________ Lane B. Mills, PhD COMMITTEE MEMBER:________________________________________________________ William Grobe, EdD CHAIR OF THE DEPARTMENT OF EDUCATIONAL LEADERSHIP: ________________________________________________________ William A. Rouse, Jr., EdD DEAN OF THE GRADUATE SCHOOL: _______________________________________________________ Paul Gemperline, PhD DEDICATION I dedicate this dissertation to my loving mother who passed away a few months ago and never got to hear me say the words, “It’s finished.” My mother was loved by many and was a very supportive mother and “Nana”. I know she was proud of me. It was that pride that kept me focused on finishing what I started. I know that’s what she expected. I miss her every day. I miss our conversations and her sense of humor. She was able to laugh and make others laugh regardless of the obstacles life sent her way. She influenced me in ways she’ll never know. She always taught me to believe in miracles, even when I doubted. She was always there for me. She will be in my thoughts on the day I walk across the graduation stage, another miracle she believed in from the start. One day when I was feeling sad that my mom would not be there to watch me complete this journey, a dear friend reminded me that she will be there and that she will have the best seat in the house. I know my mom and my brother, Robbie, will be cheering me on from the balconies in heaven. My dear mother, this is for you! ACKNOWLEDGEMENTS Years ago my parents told me they would support me as long as I wanted to attend college. Little did they know, thirty years later I would again be a college student. Three years ago I embarked on a journey to obtain a degree I never thought possible. There were times that I was challenged beyond what I thought I was able to endure, but there were people who encouraged me to stay focused and reminded me that quitting was not an option. I am deeply grateful to each of them. First, I must acknowledge my family. Completing this dissertation would not have been possible without their support. First, I thank my parents who began with nothing, yet provided me everything. They have always encouraged me to pursue my dreams. A special thank you to my Aunt Faye and Uncle Joe who have always supported me like I was their daughter and who have been my personal “cheerleaders.” Next, I owe a special thanks to my husband Ross, who has taken on many roles to enable me to complete this process. I also appreciate my son in law, Stephen, who would simply ask, “How’s it coming”? And finally I thank my children Ashley and Adam, who were always there to keep me focused. My goal is to always make you proud and to show you that all dreams are possible. I thank my daughter Ashley, who is just beginning her teaching career and has already touched so many lives, for her gently, “You can do this Mom.” That made me keep working. I also thank my son, Adam who has been waiting for three long years to finally call me “Dr. Mom.” His excitement kept me motivated. Thank you all for believing in me and supporting me in achieving all my dreams. A special thank you goes to Dr. Lane B. Mills for his unwavering support and sense of humor throughout this adventure. It is because of him that I began this journey. He served as my mentor and the reader for many drafts during this process. Thank you for all the times you would text me the words, “Keep writing.” You have been an inspiration to me and even more than a mentor, you have been my friend. I would like to thank my committee, for without each of you this would not have been possible. Dr. James McDowelle, my dissertation chair, who provided direction and insight from the very start of the process. I would also like to thank Dr. Art Rouse and Dr. William Grobe, who contributed time and effort to make this a valuable learning experience. Your support and feedback were greatly appreciated. Thanks to every participant in my study. I could not have completed my research without each of you. You were eager to share your thoughts with me and so willing to support my research. I learned so much from each of you. For that, I will forever be grateful. Finally, I want to express my sincere appreciation and gratitude to Delta Kappa Gamma Society International for the financial support of my research through a scholarship award. I also want to thank Eta State and the local Alpha Psi Chapter for their monetary support and constant encouragement. I will proudly wear the cords you presented me on my graduation day. It is an honor to be a part of such a wonderful group of women. TABLE OF CONTENTS Page TITLE……………………………………………………………………………………... i COPYRIGHT……………………………………………………………………………... ii SIGNATURE……………………………………………………………………………... iii DEDICATION……………………………………………………………………………. iv ACKNOWLEDGEMENTS………………………………………………………………. v LIST OF TABLES………………………………………………………………………... xiv LIST OF FIGURES………………………………………………………………………. xv CHAPTER ONE: INTRODUCTION TO THE STUDY………………………………... 1 Introduction………………………………………………………………………… 1 Complexity of Leadership………………………………………………………….. 3 Statement of the Problem…………………………………………………………... 4 Significance of the Study…………………………………………………………... 14 Purpose of Study…………………………………………………………………… 16 Research Questions………………………………………………………………… 16 Primary Research Question………………………………………………….. 16 Secondary Questions………………………………………………………… 16 Description of the School System………………………………………………….. 18 Implementation of the Grow Program……………………………………………... 20 Stage 1: Engage an External Consultant in the Development Process……... 22 Stage 2: Program Development…………………………………………….. 22 Stage 3: Identification of Program Participants…………………………….. 23 Stage 4: Implementation of Program……………………………………….. 26 Problem of Practice Research……………………………………………………… 27 Definition of Terms………………………………………………………………… 28 Organization of the Program Evaluation Study……………………………………. 29 CHAPTER TWO: LITERATURE REVIEW……………………………………………. 31 Introduction………………………………………………………………………… 31 The Shrinking Pool of Principal Candidates……………………………………….. 31 Support for Grow Your Own Leadership Programs……………………………….. 34 Leadership Development Program Design………………………………………… 36 Competency Framework…………………………………………………………… 40 Building a Candidate Pool…………………………………………………………. 47 Needs Assessment………………………………………………………………….. 50 Program Components………………………………………………………………. 50 Evaluation of Program Effectiveness………………………………………………. 51 Donald Kirkpatrick’s Four-Level Model of Evaluation……………………………. 52 Summary and Conclusions…………………………………………………………. 57 CHAPTER THREE: METHODOLOGY………………………………………………... 59 Introduction………………………………………………………………………… 59 Evaluation Methods of the Grow Program………………………………………… 60 Design of the Evaluation Study……………………………………………………. 61 Selection of Evaluation Study Participants………………………………………… 62 Program Evaluation Model………………………………………………………… 63 Data Collection Process……………………………………………………………. 63 Phase 1: Monthly Assessment (Quantitative and Qualitative)……………… 64 Phase 2: Summative Questionnaire (Quantitative and Qualitative)………… 64 Phase 3: GROW Evaluation (Quantitative)………………………………… 65 Phase 4: Participant Interviews (Qualitative)……………………………….. 65 Research Questions………………………………………………………………… 66 Level 1: Reaction…………………………………………………………… 67 Level 2: Learning…………………………………………………………… 68 Level 3: Behavior…………………………………………………………… 68 Level 4: Results…………………………………………………………….. 69 Data Analysis Procedures………………………………………………………….. 70 Summary…………………………………………………………………………… 75 CHAPTER FOUR: FINDINGS………………………………………………………….. 78 Introduction………………………………………………………………………… 78 Description of the GROW Program………………………………………………... 79 Characteristics of Study Participants………………………………………………. 81 Data Collection…………………………………………………………………….. 83 Monthly Assessments……………………………………………………….. 84 Summative Questionnaire…………………………………………………… 85 GROW Evaluation…………………………………………………………... 85 Participant Interviews……………………………………………………….. 86 Response Rates…………………………………………………………………….. 88 Findings Based on Kirkpatrick’s Training Evaluation Model……………………... 90 Level 1 Evaluation: Reaction……………………………………………….. 90 Research question…………………………………………………….. 90 Monthly assessments…………………………………………………. 91 Theme – self-reflection………………………………………… 93 Theme – collaboration………………………………………..... 93 Theme – presenters…………………………………………….. 94 Theme – videos………………………………………………… 94 Participant interviews…………………………………………………. 95 Theme – taping…………………………………………………. 97 Theme – school leadership……………………………………... 98 Theme – teacher leadership…………………………………….. 98 Theme – self-reflection………………………………………… 99 Theme – collaboration…………………………………………. 100 Theme – interest in leadership…………………………………. 101 Reaction Summary (Emergent Themes)…………………………………….. 105 Level 2 Evaluation: Learning……………………………………………….. 105 Research question…………………………………………………….. 105 GROW evaluation…………………………………………………….. 106 Participant interviews…………………………………………………. 108 Theme – courage……………………………………………….. 110 Theme – decision making……………………………………… 110 Learning Summary (Emergent Themes)…………………………………….. 113 Level 3 Evaluation: Behavior………………………………………………. 114 Research question…………………………………………………….. 114 Summative questionnaire……………………………………………... 114 Theme – realized potential……………………………………... 117 Theme – self-confidence……………………………………….. 117 Participant interviews…………………………………………………. 118 Theme – decision making……………………………………… 120 Theme – building relationships………………………………… 121 Theme – know thy territory……………………………………. 121 Behavior Summary (Emergent Themes)…………………………………….. 123 Level 4 Evaluation: Results………………………………………………… 123 Research question…………………………………………………….. 123 Summative questionnaire……………………………………………... 124 Theme – self-reflection (question 1)…………………………… 125 Theme – leadership principles (question 1)……………………. 127 Theme – collaboration (question 1)……………………………. 127 Theme – self-reflection (question 2)…………………………… 128 Theme – leadership principles (question 2)……………………. 129 Theme – collaboration (question 2)……………………………. 130 Participant interviews…………………………………………………. 132 Review of District Employment Data……………………………………….. 136 Results Summary (Emergent Themes)……………………………………… 136 Impact of GROW Program (Emergent Themes)……………………………. 137 Summary…………………………………………………………………………… 140 CHAPTER FIVE: SUMMARY, CONCLUSIONS AND RECOMMENDATIONS…… 141 Introduction………………………………………………………………………… 141 Summary…………………………………………………………………………… 141 Findings…………………………………………………………………………….. 144 Conclusions………………………………………………………………………… 146 Recommendations for Programmatic Improvement and Support………………….. 147 Limitations…………………………………………………………………………. 149 Recommendations Further Research………………………………………………. 150 Concluding Remarks……………………………………………………………….. 151 REFERENCES……………………………………………………………………………. 154 APPENDIX A: LETTER OF SUPPORT FROM DR. LANE B. MILLS SUPERINTENDENT OF CRAVEN COUNTY SCHOOLS…………………………….. 163 APPENDIX B: VITAE OF PRIMARY CONSULTANT……………………………….. 165 APPENDIX C: COMPETENCY FRAMEWORK………………………………………. 168 APPENDIX D: BROCHURE……………………………………………………………. 169 APPENDIX E: EMAIL CALLING FOR APPLICATIONS…………………………….. 171 APPENDIX F: GROW APPLICATION………………………………………………… 172 APPENDIX G: GROW ROSTER……………………………………………………….. 174 APPENDIX H: GROW ACCEPTANCE LETTER……………………………………… 175 APPENDIX I: GROW INTENT TO PARTICIPATE…………………………………… 176 APPENDIX J: INFORMED CONSENT TO PARTICIPATE IN RESEARCH………… 177 APPENDIX K: MONTHLY ASSESSMENT SURVEY………………………………... 180 APPENDIX L: GROW SUMMATIVE QUESTIONNAIRE……………………………. 181 APPENDIX M: GROW EVALUATION………………………………………………... 183 APPENDIX N: GROW PARTICIPANT INTERVIEW PROTOCOL………………….. 184 APPENDIX O: IRB APPROVAL……………………………………………………….. 187 LIST OF TABLES 1. Excerpt from Craven County Schools’ Five Year Strategic Improvement Plan……... 12 2. Summary of Data Collection Process………………………………………………… 71 3. Summary of Data Analysis…………………………………………………………… 76 4. Demographics and Background of Study Participants……………………………….. 82 5. Demographics of Interview Participants……………………………………………… 87 6. Response Rates……………………………………………………………………….. 89 7. Themes in Data by Kirkpatrick’s Level………………………………………………. 139 LIST OF FIGURES 1. Craven County Schools District survey results……………………………………….. 8 2. Bar graph showing district average for each survey question………………………… 10 3. KIPP Leadership Competency Model………………………………………………… 42 4. Observable characteristics versus underlying competencies…………………………. 43 5. Diagram of competency framework………………………………………………….. 46 6. Kirkpatrick’s Model of Program Evaluation…………………………………………. 54 7. Diagram of Donald Kirkpatrick’s Chain of Evidence………………………………… 56 8. Bar graph showing monthly assessment data results…………………………………. 92 9. Bar graph showing participant responses regarding tapping……….………………… 96 10. Bar graph showing participants’ satisfaction with GROW Program…………………. 104 11. Bar graph showing pre- and post-evaluation results………………………………….. 107 12. Bar graph showing growth by leadership principle………………………………....... 109 13. Bar graph showing impact of grow program on increasing the pool of qualified candidates for leadership positions……………………………………….................... 112 14. Bar graph showing change in interest in leadership………………………………….. 116 15. Bar graph showing level of interest in leadership after program participation………. 119 16. Bar graph showing participant opinions regarding contribution of the GROW Program to a stronger pool of applicants for school leadership positions……………. 126 CHAPTER ONE: INTRODUCTION TO THE STUDY Introduction According to research conducted by the Wallace Foundation, “Leadership is second only to classroom instruction among all school-related factors that contribute to what students learn at school” (Leithwood, Louis, Anderson, & Wahlstrom, 2004, p. 1). While there is no question that leadership impacts schools, finding quality leaders that can drive school improvement continues to be a challenge. An increase in accountability, higher demands for leadership, budget cuts, and school reform efforts have contributed to the shortage of teachers willing to move into leadership positions, making recruitment efforts even more difficult (Grubb & Flessa, 2006; Guterman, 2007; Whitaker, 2001). The No Child Left Behind Act of 2001 (NCLB) requires high quality leaders that have a positive impact on school performance and proposes sanctions for school leaders whose schools are low-performing, including removing the principal (Guterman, 2007). These threats relative to performance expectations also negatively impact the interest level in school leadership positions (Guterman, 2007). In addition to the low number of applicants interested in school leadership, there is also concern regarding the quality and preparation of the few educators that are potential candidates for school leadership positions (Whitaker, 2001). While there has been a focus on accountability associated with school leadership, there are many factors that make the principalship less than attractive as a profession for educators (Fuller, 2009). “The principalship is the kind of job where you’re expected to be all things to all people” (Fullan, 2001, p. 23). In an article discussing the exodus of school principals, Jimmy Guterman (2007) wrote: Aside from the usual demands of educational leadership and management, the principal serves as the school's public face and spokesperson, and must respond to parents, 2 teachers, unions, and many other constituencies. In addition, he or she is in charge of truly mundane things, like keeping the lights on. (p. 1) The increased demands and growing expectations regarding accountability impacts the dropout rate for principals and makes recruiting school leaders even more difficult. A study conducted on the tenure and retention of newly hired principals revealed that ninety percent of principals that leave schools, leave the principalship altogether (Fuller, 2009). The demands of the principalship are causing principals to retire early and teacher leaders are “concluding that it is simply not worth it to take on the leadership of schools” (Fullan, 2001, p. 5). Without those who desire to become school leaders, school districts will be in a state of crisis relative to filling the impending vacancies due to retirements unless innovative and effective measures are taken to recruit, develop, train, and support school leaders. While this problem is not new, few school districts have clear succession plans in place for the purpose of attracting, developing, and supporting school leaders (Wilson, 2009). Districts are not developing avenues to create a pipeline to the principalship (Guterman, 2007). Guterman (2007) suggests the development of a structured and systematic plan that provides a sequence of programs developed to support the recruitment and retention of principals over time. It is only when the plan is developed with the goal of “having the right people at the right places at the right time to do the right things and get the right results” that succession planning in a district will become a systemic and successful process (Rothwell, 2001, p. 15). Andy Hargreaves stated, “One of the most significant factors affecting the life of a school and sustainability of its improvement efforts…is leadership succession” (Hargreaves & Fink, 2002, p. 45). A review of the literature suggests that succession planning relative to school leadership has been discussed for years, but few school districts have been successful in fully 3 implementing successful succession plans that prepare for the seamless transition of leadership within the district (Canavan, 2001; Lacey, 2003; Sousa, 2003; Wilson, 2009). Most school districts claim to have succession plans in place, but few have all the components research suggests is needed to provide the instructional leaders needed in schools. The multifaceted plan should include more than just a process to pick the next leader of a school. According to Hargreaves and Fink (2006), “Effective succession means having a plan and making plans to create positive and coordinated flows of leadership, across many years and numerous people” (p. 92). The plan must include a variety of components addressing recruitment, preparation, selection, and training in a seamless manner in order to have powerful instructional leaders prepared for the dynamic leadership needed in today’s schools. John Daresh (2004) suggests that succession plans must go well beyond hiring decisions and include training for principals that is aligned with leadership standards with emphasis on mentoring and professional learning communities. Leadership succession planning must be an integrated and systematic approach in order to identify, recruit, and develop quality leaders (Chapman, 2005). In order to address the school leadership crisis, school districts must have research based succession plans in order to deepen the pools of leadership talent with the skills and abilities to address the complexities of school leadership (Daresh, 2004; Fullan, 2001). Complexity of Leadership Leadership is more than just running a school. School leadership is a complex position that requires a multifaceted approach to problem solving. According to Fullan (2001), school leadership is complex due to four specific reasons: 1. the new challenges of school leadership requires more effort and deeper knowledge than ever before; 4 2. as a dynamic decision maker, the school leader must be able to address many problems at once; 3. implementing change requires multiple actions by a dynamic and skilled leader; and 4. school leadership is not an easily designed step by step process that can be accomplished by following a prescribed checklist. School districts must consider these complexities in developing a succession plan for future leaders. “The old world is still around with the expectations to run a smooth school, and to be responsive to all; simultaneously the new world rains down on schools with disconnected demands expecting that at the end of the day the school should be constantly showing better test results, and ideally becoming a learning organization” (Fullan, 2001 p. 193). Statement of the Problem Effective school leaders are essential to school and district success, yet finding strong instructional leaders prepared to lead schools in the twenty-first century is more challenging than ever before. Increased accountability and more demands make the school principalship less attractive (Grubb & Flessa, 2006; Guterman, 2007; Whitaker, 2001). Based on current data provided by Ervin Patrick, Director of Human Resource Services Division of Craven County Schools, Craven County Schools has the challenging task of filling a high number of positions within the next few years due to administrative retirements (E. Patrick, personal communication, October 25, 2013). From 2010 to 2013, the principal turnover rate in Craven County Schools was twenty-two percent. In the last eighteen months alone, twenty-nine leadership positions have been filled. Based on the experience level of current administrators, 54% of principals will be eligible to retire in five or less years. Eighty-one percent of principals will be eligible to retire in less than 5 ten years. According to district data provided by Robert Kelley, Licensure Specialist in Craven County Schools, there are eleven certified principals not currently in a leadership role (R. Kelley, personal communication, October 25, 2013). Of those eleven, only four have expressed an interest in moving into a leadership position. Two additional certified principals were in administrative positions and requested to return to the classroom. Thirteen teachers are currently enrolled in programs that lead to principal licensure. This data reveals an alarming gap between the need for school administrators and the anticipated supply of qualified candidates preparing for and/or interested in serving in leadership roles in Craven County Schools. If this trend continues with currently licensed principals and those enrolled in leadership preparation programs, the gap will only get wider (E. Patrick, personal communication, October 25, 2013). Research indicates that student performance is impacted by principal effectiveness (Barth, 2001; Branch, Hanushek, & Rivkin, 2012; Coelli & Green, 2012). This impact was affirmed in Craven County Schools based on results of a district survey sent to both certified and classified staff members. Craven County Schools deployed a district-wide employee survey in order to seek feedback regarding the effectiveness of school principals. Responses from staff provided evidence that teachers and other staff members believe the school principal has an impact on both student performance and teacher practice. Results indicate that teachers and other school staff members value instructional leadership as critical to the work of the school principal. In order to determine principal effectiveness based on staff input, the following statements were included in the district survey: • (Q 1) Teachers have time available to collaborate with colleagues. • (Q 2) Teachers are encouraged to participate in leadership roles. 6 • (Q 3) School administrators support teachers’ efforts to maintain discipline in the classroom. • (Q 4) Teachers have sufficient instructional time to meet the needs of all students. • (Q 5) There is an atmosphere of trust and mutual respect in this school. • (Q 6) Teachers feel comfortable raising issues and concerns with the school leader. • (Q 7) Teachers receive feedback that can help them improve practice. • (Q 8) Teachers work in professional learning communities to develop and align instructional practices. • (Q 9) Overall, my school is a good place to work and learn. Staff members were asked to review each statement and rate the statement’s accuracy based on a likert-type scale in order to measure the intensity of agreement with each statement. The district survey was comprised of a five point scale allowing each school staff member to rate how much they agreed or disagreed with each statement. In order to improve the accuracy of the feedback, there was the option to select “don’t know/na” if the item did not apply to the staff member being surveyed. The likert-type scale used the following response options: • Strongly Disagree • Disagree • Agree • Strongly Agree • Don’t Know/NA From the results, a weighted system was used to give a value to each response. Those values were then equated to a percentage to serve as a numerical representation of the staff level of agreement. Finally, all areas were averaged together to determine the percentage of staff 7 members that felt current leadership was effective in supporting the instructional practices of teachers and positively impacting student performance. Survey results are depicted in Figure 1. Based on the survey results, many of the current school leaders in Craven County Schools are not perceived to be highly effective in performing the role of an instructional leader to the level teachers and staff members believe is needed to support teaching and to have a positive impact on student achievement. Only six of the twenty-five schools rated their principal at 90% or higher. Eleven school principals were rated between 80% and 90% effective. Six principals were rated between 70 and 80% effective. Two principals were rated below 70% effective. Therefore, more than 32% of schools have less than 8% of staff members who feel the principal is effective. This data serves as evidence that current school leaders are not meeting the expectations of teachers and staff members in the areas of instructional leadership and teacher support. Interestingly enough, the two schools that rated the principal below 70% have dropped in student performance for the past two years and have experienced an increase in teacher turnover (E. Patrick, personal communication, October 25, 2013). The school with the lowest school performance has 63% of the staff that felt leadership was effective in meeting the current demands of the role of principal. The elementary school with the highest percent of teachers (95%) that felt the leadership was effective in providing instructional leadership and teacher support also has the highest student performance in the district at 95% cumulative student proficiency. These data points suggest a link between principal leadership, teacher satisfaction, and student performance. While there are limitations with the validity due to staff self-reporting, the data suggests that previous district initiatives designed to recruit, train, develop, and support school leaders have not been effective in adequately preparing principals for the role of instructional leader. 8 Q 1 Q 2 Q 3 Q 4 Q 5 Q 6 Q 7 Q 8 Q 9 AVERAGE ELEM 1 64.52% 66.67% 100% 67.74% 87.10% 59.38% 68.75% 90.63% 78.79% 75.95% ELEM 2 75% 72.22% 100% 86.11% 86.11% 80.56% 77.78% 94.59% 94.29% 85.18% ELEM 3 65% 71.43% 94.74% 94.74% 100.00% 80.95% 95.24% 100.00% 100.00% 89.12% ELEM 4 73.68% 57.89% 92.31% 77.78% 79.49% 70.00% 65.00% 92.31% 95.00% 78.16% ELEM 5 68.75% 59.57% 95.35% 70.83% 91.30% 63.04% 80.43% 86.96% 95.56% 79.09% ELEM 6 85.29% 70.59% 100.00% 97.30% 88.89% 77.78% 91.67% 100.00% 97.30% 89.87% ELEM 7 93.10% 75.86% 100.00% 93.10% 96.55% 89.66% 89.66% 100.00% 100.00% 93.10% ELEM8 76% 79.17% 100.00% 79.17% 91.67% 79.17% 87.50% 95.65% 91.30% 86.63% ELEM 9 85.37% 82.93% 100.00% 100% 100.00% 97.56% 97.56% 95.12% 100.00% 95.39% ELEM 10 74.29% 51.43% 100% 91.43% 96.88% 84.38% 91.18% 96.88% 100.00% 87.39% ELEM 11 48.48% 48.48% 96.67% 78.17% 96.77 65.63% 75.00% 96.88% 78.79% 76.09% ELEM 12 59.26% 38.36% 92.31% 70.37% 73.08% 30.77% 42.31% 80.77% 77.78% 62.78% ELEM 13 45.45% 48.57% 93.94% 69.70% 64.71% 57.14% 62.86% 91.43% 77.14% 67.88% ELEM 14 66.67% 70.83% 97.67% 93.33% 90.91% 88.37% 88.64% 95.65% 100.00% 88.01% ELEM 15 63.64% 48.48% 100.00% 78.13% 96.88% 87.88% 90.91% 90.91% 96.88% 83.75% MID 1 75.61% 60.98% 95.12% 92.11% 92.68% 82.93% 82.93% 95.00% 95.12% 85.83% MID 2 76.92% 67.31% 94.12% 82.35% 88.46% 72.00% 88.24% 94.00% 89.80% 83.69% MID 3 96.55% 96.67% 100% 96.67% 100.00% 90.32% 90.32% 100.00% 96.55% 96.34% MID 4 87.50% 81.82% 93.94% 87.88% 97.06% 100.00% 96.97% 90.91% 100.00% 92.90% MID 5 81.25% 70.21% 87.50% 68.75% 68.75% 70.83% 55.10% 91.30% 82.61% 75.14% HS 1 100% 100.00% 100% 100% 100.00% 100.00% 100.00% 100.00% 100.00% 100% HS 2 85.71% 100.00% 100% 100% 100.00% 100.00% 100.00% 100.00% 100.00% 98.41% HS 3 71.91% 73.86% 98.83% 65.88% 81.40% 74.42% 86.05% 89.29% 95.06% 81.85% HS 4 53.47% 68.37% 87.13% 51% 72.12% 67.65% 64.71% 90.38% 92.23% 71.90% HS 5 76.06% 82.09% 85.07% 76.06% 88.06% 88.06% 83.82% 86.76% 84.85% 83.43% DISTRICT 71.89% 68.72% 94.48% 78.54% 86.71% 75.40% 80.14% 84.96% 92.22% 81.45% Green is greater than 90% Yellow is between 70% and 90% Red is less than 70% Note. Adapted from Craven County Schools’ District Survey Results. Craven County Schools by L. B. Mills, n.d. Retrieved from http://www.craven.k12.nc.us/?page_id=371. Copyright 2013 by Craven County Schools. Reprinted with permission. Figure 1. Craven County Schools District survey results. 9 The survey averages support the need for additional initiatives to build leadership capacity in Craven County Schools, as illustrated in Figure 2. District survey averages yield only two areas that meet the district goal of school leaders averaging at or above ninety percent relative to school-level leadership satisfaction. Interestingly enough, the area that ranks lowest at 68.7% is the question related to opportunities for teachers to lead. This trend is in direct alignment with this study and the problem of practice it intends to address, the evaluation of the implementation of a Grow Your Own teacher leadership development program in Craven County Schools in order to deepen the candidate pool for future school leadership positions. The data suggests teacher leaders are not developed at the school level. Based on these statistics, the superintendent of Craven County Schools, Dr. Lane B. Mills, initiated the development of a comprehensive succession plan that included researched-based strategies proven effective in recruiting, preparing, training, and supporting school principals in order to maintain school district momentum and continuous improvement initiatives (L. B. Mills, personal communication, October 25, 2013). At the time of this study, Craven County Schools Superintendent, Dr. Lane B. Mills, has been in the position for eighteen months and during that time has been confronted with the challenge of filling twenty-nine leadership positions within the district. Based on several interviews with Dr. Mills, it is evident that securing and maintaining a qualified pool of interested candidates is difficult. In addition, the limited pool of candidates within the district, along with the pending retirements and potential advancement of other leaders on the horizon, places the district in a state of crisis. Thus the need for the development and implementation of a researched-based succession plan is critical. Dr. Mills shared concerns regarding recruiting, developing, and supporting potential leaders in order to deepen the pool of internal applicants for 10 Note. Adapted from Craven County Schools’ District Survey Results. Craven County Schools by L. B. Mills, n.d. Copyright 2013 by Craven County Schools. Reprinted with permission. Figure 2. Bar graph showing district average for each survey question. 0% 20% 40% 60% 80% 100% DISTRICT SURVEY AVERAGES Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 AVERAGE 11 developing, and supporting potential leaders in order to deepen the pool of internal applicants for school leadership positions in Craven County Schools. He anticipates this study will provide evaluation data relative to district succession planning initiatives that can, in turn, be presented to the Craven County Board of Education in order to garner support for the implementation of innovative programs designed to identify, grow and support future leaders in the district. The Craven County Board of Education requires the superintendent’s cabinet to develop, deploy, continuously monitor, and consistently revise a strategic plan that drives district initiatives (L. B. Mills, personal communication, October 25, 2013). Due to the problem the district faces relative to filling leadership positions within the district, the five year strategic plan for Craven County Schools was revised and approved by the Board of Education on August 22, 2013 to include a specific goal related to 21st Century Professionals (L. B. Mills, personal communication, October 25, 2013). Specifically, the goal is to create a culture that attracts, supports, and retains high-quality staff (L. B. Mills, personal communication, October 25, 2013). The strategic plan includes key strategies to support goal attainment. The basis of this program evaluation study is to determine if the key strategies implemented regarding creating and implementing a comprehensive plan to recruit and develop leaders that possess the skills to support student learning are effective. The key strategies in Craven County Schools’ Five Year Strategic Improvement Plan relative to highly effective staffing are noted in Table 1. Superintendent Mills asked Wendy Miller, Assistant Superintendent of Human Resource Services and the evaluator in this study, to complete a program evaluation specific to the implementation of a Grow Your Own leadership program in Craven County Schools. The data will be collected relative to the twenty-five participants that completed the GROW Program. Participants will be selected from across the district with representation from elementary schools, 12 Table 1 Excerpt from Craven County Schools’ Five Year Strategic Improvement Plan Goal Strategy Goal 7: Highly Effective Staffing – Craven County Schools will create a culture that attracts, supports, and retains high-quality staff. Key Strategy: The district will develop, implement and monitor a professional development plan to ensure that all “leaders” possess the skills to support student learning. Key Strategy: The district will create and implement a comprehensive plan to develop, recruit and employ teachers and leaders to reflect the diversity of the student population. Key Strategy: The district will provide support to ensure all staff meets the federal definition as Highly Qualified and state licensure requirements. Key Strategy: The district will consistently implement and monitor the evaluation process and procedures for certified and non-certified staff. Note. Adapted from Craven County Schools’ Five Year Strategic Plan. Craven County Schools by L. B. Mills, n.d. Retrieved from http://www.craven.k12.nc.us/?page_id=367. Copyright 2013 by Craven County Schools. Reprinted with permission. 13 middle schools, traditional high schools, and early college high schools. The study will focus on qualitative and quantitative data from program participants including surveys and personal interviews. Specifically, Dr. Mills is interested in data to determine if district initiatives designed to recruit, prepare, train, and support future district leaders had a positive impact on program participants as stated in his letter of support for this study (see Appendix A). The study will answer the following key question: Will the design and implementation of a district level teacher leadership development program increase the pool of qualified and competent leaders in Craven County Schools? It is the intention of the study to evaluate the Grow Your Own leadership program designed and implemented in Craven County Schools. This program evaluation will determine if program implementation had a positive impact on participants and addressed the problem of the lack of qualified candidates currently within the district. The evaluator is also interested in determining if the completers of the district-designed Grow Your Own leadership program had an increased level of interest in securing school leadership positions. Long term evaluation measures are aimed at determining if program participants that move into leadership roles are able to demonstrate their effectiveness by making a positive impact on student performance, staff satisfaction, and teacher retention within their assigned schools. This study is designed to evaluate a potential solution to the problem of the urgent need for the development of quality school-based leaders implemented in Craven County Schools through a Grow Your Own leadership development program. Data will be analyzed relative to participants in the district leadership program in order to determine if the deployment of a district leadership development program was successful. The results of this study will be shared with 14 the Superintendent’s Cabinet and ultimately the Craven County Board of Education in order to determine if the Grow Your Own leadership development program implemented in the district should be an ongoing component of a comprehensive succession plan aimed at recruiting, preparing, training, and supporting school leaders in Craven County Schools. Significance of the Study The significance of this study is to provide research addressing a problem of practice relative to leadership development in Craven County Schools. The study will examine and evaluate one school district’s efforts to develop and implement a Grow Your Own leadership development program designed to address the issue of the shortage of qualified candidates to fill current and future leadership positions. Specifically, this study is designed to determine if those leaders identified, recruited, prepared, trained and supported through the implementation of research-based programming, are better prepared to take leadership positions in the schools. The evaluator is also interested in determining if the completers of the district-designed, Grow Your Own leadership program have an increased level of interest in securing school leadership positions. A program evaluation model will be used to determine the success of the implementation of a district succession plan, specifically a Grow Your Own teacher leadership academy designed to identify, recruit, and develop future school leaders. Summative program data will be analyzed to determine program impact and effectiveness. The methodology for this study utilizes a mixed model that includes quantitative and qualitative data designed to evaluate the success of the implementation of a research-based Grow Your Own leadership development program. The purpose of a mixed methodology approach to evaluation is to gain rich data from multiple sources. A holistic approach will be used to 15 evaluate participant reactions, acquired learning, application of learning, and behavioral changes based on participation in the program. Data to determine the level of success of the implementation of the leadership program will include: • Monthly participant assessment of program components • Summative evaluation through participant questionnaires • Evaluation using a retrospective post-then-pre assessment 7-point Likert-type scale • Interviews with selected participants using a semi-structured interview protocol in order to gain a deeper understanding of the short and long term benefits of the program Analysis of the data collected will be used to determine if district initiatives designed to identify, recruit and train future school leaders had a positive impact on the development and interest level relative to serving as a future school leader. Data collected regarding successful initiatives will be used to address the district problem of the lack of qualified candidates interested in leadership positions in Craven County Schools. Data will be analyzed using Donald Kirkpatrick’s (2006) model of program evaluation, which includes: • Level 1: Reaction - How well did the learners react to the leadership program? • Level 2: Learning - To what extent did the program participants change attitudes, improve knowledge, and/or increase skill level? • Level 3: Behavior – What changes in behavior occurred in program participants? • Level 4: Results – What are the results of the learning acquired in the leadership program to the individual and the organization? 16 Purpose of Study The purpose of this study is twofold. First, the purpose of the study is to explore the implementation of a researched-based leadership development program in Craven County Schools aimed at the development of teachers as future school leaders. In addition, the study will evaluate the success of the school system's efforts to develop and implement a Grow Your Own leadership development program based on providing more qualified and interested candidates in school leadership positions. Research findings will be used to determine if the components deployed in the implementation of the Grow Your Own district succession plan had a positive impact on program participants and solved the problem of practice relative to a shortage of qualified school leadership candidates. The results of this study will provide research helpful in the continuous improvement process of leadership development in this system and may be utilized by other school districts coping with the issue of building leadership capacity within the district. Research Questions Primary Research Question • How effective is a school system’s research-based Grow Your Own leadership development program in growing a pool of qualified candidates to fill future leadership positions? Secondary Questions 1. Did the Grow Your Own leadership program meet the needs of the participants? 2. What did participants in the Grow Your Own leadership program learn relative to leadership principles? 3. How has the behavior and attitudes of the participants in the Grow Your Own 17 leadership program changed relative to school leadership as a result of participating in the program? 4. Did the Grow Your Own leadership program meet the goal of designing and implementing a program effective in increasing the pool of qualified and competent leaders in Craven County Schools? In order to address the research questions as outlined above, a review of the relevant is necessary to determine the following: • individual components of a Grow Your Own succession plan most effective in developing teachers to become school leaders • process used to select a pool of qualified participants in a Grow Your Own leadership program • most appropriate models to evaluate program effectiveness By conducting a thorough review of the literature relative to research-based components of Grow Your Own leadership programs and ways to evaluate such an initiative, this study will yield the data for Craven County Schools’ district leaders to consider in the development, implementation and evaluation of a teacher leadership program as part of a district succession plan. The work of Hartle and Thomas (2003) suggests a six step approach to succession planning which includes: 1. Creating a culture that is committed to growing future leaders. 2. Auditing the current and future needs for leadership. 3. Developing a systematic approach to leadership by gaining consensus on the vital leadership characteristics needed to effectively lead schools. 4. Identify potential leaders and design programs to foster their continued growth. 18 5. Monitor the development of future leaders and gain feedback on the professional development activities needed in order to provide those needed opportunities for growth as a leader. 6. Continue to support leadership growth by providing leadership networking opportunities and mentoring support for new leaders. The results of this program evaluation will be used to determine which of the six areas need additional attention by district leaders in the development of a formal district succession plan. This study will also use evaluation data to determine the support and development teachers require in order to heighten their interest in pursuing school leadership positions. Finally, the findings will be used to improve the current Grow Your Own program initiatives in Craven County Schools. Description of the School System Located in eastern North Carolina, about forty miles from the coast, Craven County Schools is comprised of twenty five schools covering approximately seven hundred twelve square miles. Out of 115 school districts, Craven County Schools ranks 42 in the state relative to student performance based on composite scores (North Carolina Department of Public Instruction, 2013). Based on an Education Value-Added Assessment System (EVAAS), six schools in the district exceeded growth and eleven schools met growth (North Carolina Department of Public Instruction, 2013). There are fifteen elementary schools, five middle schools, three traditional high schools and two early colleges. The district serves over 14,000 students. Of those students, less than 1% are American Indian, 2% are Asian, 6% are Hispanic, 36% are African American, and 56% are white (Craven County Schools, 2013). Other demographic information includes 58.5% of students qualify for free and reduced meals, 12.4% 19 of students are identified as having special needs, 8.3% of students are identified as academically gifted, and 4.2% of students are receiving English as a second language services (Craven County Schools, 2013). The district four-year cohort graduation rate of 86% continues to be higher than the state graduation cohort rate of 82.5%, with Craven Early College leading the high schools in Craven County at 95% (Craven County Schools, 2013). Craven County Schools employs approximately 980 certified teachers, of which 6% have National Board Certification. Currently, 98.8% of Craven County teachers are highly qualified in core content areas as defined by federal No Child Left Behind standards (Craven County Schools, 2013). There is an average annual teacher turnover rate of 16% (North Carolina Department of Public Instruction, 2013). The district currently has twenty-five principals, and thirty-three assistant principals. Sixteen percent of principals have completed an advanced college degree beyond a master's degree. The principal turnover rate is 22%, which is more than double the state average of 10% (North Carolina Department of Public Instruction, 2013). The Craven County School System has three distinct areas of service. There is a western area that is more rural with little industry, an urban area with a historical district and local tourist attractions, and an eastern area that is primarily military. These unique areas, and their tendency for natural attrition, provide challenges for district administrators to recruit and retain school leaders. In fact, in the last eighteen months alone, twenty-nine leadership positions have been filled (Craven County Schools Human Resource Services Division, 2013). Based on the experience level of current administrators, 54% of principals will be eligible to retire in five or less years and 81% of principals will be eligible to retire in less than ten years (Craven County Schools Human Resource Services Division, 2013). This data supports the need for high quality candidates interested in leadership positions necessary to maintain school district momentum and 20 continuous improvement initiatives (Craven County Schools, 2013). An analysis of this data by the superintendent’s cabinet determined the need to design and implement a leadership development program aimed at increasing the number of qualified candidates interested in pursuing school leadership positions. Dr. Lane B. Mills, superintendent of Craven County Schools charged the human resource services division with the tasks of developing a Grow Your Own leadership program. In addition, he requested from the evaluator in this study, data regarding the success of the program in developing a quality pool of school leadership candidates. Implementation of the Grow Program The Human Resource Services Division researchers included the Director of Human Resource Services, Ervin Patrick, and the Assistant Superintendent of Human Resource Services and the evaluator in this study, Wendy A. Miller. A review of the current literature was conducted relative to the required components of a successful leadership development program to aid in the planning and implementation of a Grow Your Own leadership development program in Craven County Schools. The literature suggested a comprehensive plan including initiatives designed to recruit, train, and support potential leaders in order to deepen the pool of leadership talent (Daresh, 2004). Specifically, the review of the literature was aimed at determining practices most effective in Grow Your Own leadership programs in order to develop and sustain leadership in a local school district. Barth (2001) identified four essential questions when developing and deploying a district succession plan: 1. How do you identify and recruit those who have the characteristics to become effective principals? 21 2. What steps can be employed to encourage those potential leaders to pursue the principalship? 3. After potential school leaders have been identified and recruited, what measures can be taken to prepare them for the challenging role of a school leader? 4. What can be done to support and sustain the leader once they are in the role? Based on a review of the current literature, a locally designed leadership development program was implemented in order for participants to aide future leaders to Gain skills, Realize potential, Optimize Impact on Others, and increase Work Satisfaction (GROW). The review of the research led to the development of the GROW program, a teacher leadership development program. This program was designed to address facilitating and building the leadership skills of teacher leaders as a part of a multifaceted approach to succession planning developed for the purpose of the identification, recruitment, and development of future school leaders. The objective of the GROW program was to develop leaders from within the organization while deepening the pool of quality candidates as a future source for school administrators. The primary function of the program was to equip teacher leaders with the insights, knowledge and skills essential to sustaining systemic improvements for the short and long term organizational goals of developing effective school leaders. The evaluator in this study was charged by Superintendent Lane B. Mills with the task of developing and implementing the program and monitoring the success of the program through researched based program evaluation methods. The recommendations of Bumphus and Royal (2008) were used as a guide for program development. Based on a review of the research by Bumphus and Royal (2008), the following stages of implementation occurred: 22 Stage 1: Engage an External Consultant in the Development Process The consultant selected for collaboration and implementation of the leadership development program was The Masonboro Group. This consultant firm specializes in education, non-profit, and governmental applications. A team of seasoned public school, governmental and private sector veterans who possess skills in the area of leadership development, evaluation, policy analysis, assessment, professional development and other organizational improvement areas are firm consultants. The expertise of the consultant, was used to customize the GROW program to meet the needs and align with the strategic plan of Craven County Schools. The vitae’ of the primary consultant on the development and implementation of the GROW Program is included (see Appendix B). Stage 2: Program Development According to Bumphus and Royal (2008), leadership development programs must have a foundation based on the competencies of successful leaders and include the components of successful leadership programs. The history behind the problem of practice in this study was communicated and discussed with the superintendent’s cabinet and the external consultant. The Urban Excellence Framework, developed by New Leaders for New Schools (2011), was reviewed to ensure the program components were research-based and designed to develop the next generation of school leaders with the skills necessary to drive school improvement. Research supports a competency framework to align training with system directions and leadership needs (Cheney, Davis, Garrett, & Holleran, 2010). After a thorough literature review, a competency framework was developed based primarily on the research recommendations of New Leaders for New Schools (2011) and Cheney et al. (2010). The framework defined the skills, knowledge and dispositions that effective schools 23 leaders must develop in order to drive change and positively impact student achievement and was developed based on the vision, mission, strategic directions, and core values of Craven County Schools (see Appendix C). The framework was used to guide program development, detail elements of the program and provide the foundation used to evaluate program outcomes. In addition, the framework was utilized as the basis on which the candidate pool was recruited, selected and trained. Research conducted as a part of this study was shared with the consultant and used to develop program goals, implementation strategies and evaluation methods. Program dates, topics and criteria for participant selection were determined. Stage 3: Identification of Program Participants The goal of any leadership development program should be to not only prepare more school leaders, but the right ones (Wallace, 2011). The first step in the process was to develop criteria to be used in the recruitment and selection of GROW participants. Based on a needs assessment defined by the Superintendent’s Leadership Cabinet and research conducted by Cheney et al. (2010) and Byham, Smith, and Paese (2002), the criteria was developed to determine qualification requirements of program participants. The selection criteria developed included participants who: • Had a minimum of three years of successful teaching experience and at least ten years from retiring • Possessed peer credibility as an effective teacher • Had leadership experience • Supported the values and strategic directions of Craven County Schools • Exhibited excellent communication and interpersonal skills 24 • Communicated effectively in writing • Had the potential to develop skills in collegial coaching and mentoring Since the literature supports the role of current administrators in helping the school district identify a potential candidate pool, the recruitment plan involved principal tapping of those potential candidates in order to cultivate their interest in program participation (Grunow, Horng, & Loeb, 2010). Based on the work of Cheney et al. (2010), Grunow et al. (2010), and Byham et al. (2002), the following strategic recruitment initiatives were employed: • The recruitment and selection was based on specific and transparent criteria aligned with the competency framework and shared with potential participants and school leaders. • The selection was competitive. • Recruitment efforts attempted to generate a diverse pool of potential candidates relative to race, gender, school level, and experiences. • Specific recruitment strategies included flyers, email blasts, website announcements, and principal tapping. Recruitment began by distributing a brochure calling for applications from interested candidates that included the selection criteria (see Appendix D). An email was sent to all district employees introducing the teacher leadership program (GROW) and calling for applications (see Appendix E). The introduction included the purpose of the program to cultivate the next generation of school leaders by identifying, recruiting, and training educators who share the commitment of Craven County Schools to make a positive difference in education through positive leadership practices. The program offered an opportunity to develop those interested in school leadership while offering networking opportunities with other passionate educators. 25 Interested candidates were required to return a completed application to the Human Resource Services Division. The application (see Appendix F) consisted of demographic information along with ten short response questions which included information relative to the following areas: • Reasons for pursing a position in the Teacher Leader Program • Teaching experience • Degrees and certifications held • Leadership roles currently held in the school • Description of learning from serving in leadership roles • Willingness to complete assessment instruments that will help assess leadership skills, strengths and areas for improvement • Description of strengths and weaknesses as a leader • Reflection on their principal’s assessment of their strengths and weaknesses • Commitment to professional study beyond the regular school day in order to fulfill program requirements • Assessment of principal recommendation for participating in the program Once applications were received, principal recommendations were required based on program criteria and the applicant’s potential as a future school leader. All program components were used to select twenty-five program participants. Diversity in the participant pool was considered including leadership and teaching experiences, school location, grade level, gender and minority representation, as noted on the roster (see Appendix G). Participants were notified in writing regarding selection to participate in the GROW program (see Appendix H). The acceptance letter shared next steps and included an “Intent to Participate Form”, which outlined 26 training dates and a statement requiring participants to commit the time and effort necessary to fully benefit from the program. A signed statement agreeing to attend all sessions and participate in activities in order to grow as a school leader was required of all potential candidates of the GROW program (see Appendix I). Stage 4: Implementation of Program Based on the research of Davis et al. (2005), program implementation included ten monthly hands-on and competency-based sessions with authentic opportunities to lead adults, make mistakes, and grow. The GROW program was designed to address the individual and collective developmental needs of designated participants as set forth in the selection criteria and in relationship to the human, conceptual and technical skills required of school leaders to sustain enhanced student performance in Craven County Schools. Based on a review of the research of Bumphus and Royal (2008) and New Leaders for New Schools (2011), the following leadership topics and practices were discussed with the consultant and determined to be necessary components of the program due to their ability to drive school improvement: • Core Values of Leadership/Communication • School Culture and Climate/Building a Culture • Organizational Development/Impact Analysis • Operations and Systems/Know Thy Territory • Change Management/Courage • Personal Leadership/ Know Thyself • Real-world experiences and challenges/Conflict Management/Decision Making • Accountability/Goal Setting • Collaboration opportunities through the development of a cohort group 27 Problem of Practice Research There has been ongoing debate regarding appropriate doctoral program requirements within educational leadership programs. Specifically in question is the relevance of dissertation design in Doctorate of Philosophy (PhD) as compared to Doctorate of Education (EdD) programs. Shulman (2004) suggests further distinction between a PhD and EdD by stating, “We need PhD preparation for scholarship and EdD preparation for practice.” Herein lies the vision for current EdD programs to make learning and research meaningful to practice. The greatest skill educational leaders can master is the ability to solve problems regardless of the leadership level in which they are serving (Shulman, 2004). Therefore, this study will focus on a real and meaningful problem of practice. By addressing authentic issues, education practitioners will learn to use research to assist in solving a current problem of practice in education. The emphasis will be on solving a complex problem while learning to be a reflective practitioner who is constantly seeking new and innovative ways to improve educational practices through research and study. Based on recent research, many universities have evaluated and made changes to curriculum and program requirements in order to meet the needs of those enrolled in doctoral programs (Everson, 2006). Programs are designed and aligned to the work the candidate with actually do upon graduation. According to Everson (2006) “the intention has been to separate the PhD program that is preparation for scholarship from the EdD program that is preparation for practice.” This study uses the approach supported by research and uses an action research model to address a current problem of practice in the field of K-12 education. This study will provide research to help improve the practices of a school district as well as help the candidate prepare 28 for future leadership challenges that will require inquiry, research, and reflection to make a difference in school leadership and school district practices. Definition of Terms The terms used throughout this study are defined as follows: Donald Kirkpatrick’s Program Evaluation Model: A model for evaluating training programs using reaction, learning, behavior, and results. This model has been used in corporate and academic settings to evaluate program success based on research and best practices. Grow Your Own Leadership Program: A short-term program designed to recruit and train future leaders from within the organization GROW Program: A 10-month teacher leadership program developed and implemented in Craven County Schools based on the strategic goals of the district and the need to deepen the pool of qualified candidates for school leadership positions. The acronym GROW is based on the desire for participants to Gain skills, Realize potential, Optimize impact on others, and increase Work satisfaction. Likert-type Scale: A multiple-choice rating scale used to determine participant views toward a subject based on how strongly participant feels about the statement. Principal: the instructional leader of a school that selects, supervises, and evaluates all staff at the assigned site. Principal candidate: The person who is preparing, either through coursework or experience, to enter the position of school principal. Principal Tapping: The informal encouragement of a teacher by a principal to consider and pursue opportunities that lead to positions in school leadership. Principalship: the position of highest authority at the school site. 29 Program Evaluation: The systematic assessment of a program by collecting and analyzing data relative to program effectiveness based on established goals and objectives. Recruitment: The process used to attract applicants for positions within the school district. Retention: The act of retaining principals in school leadership positions. Selection: The process by which a person is chosen to assume a position within the school district based on a list of standards and expectations relative to desired leadership characteristics. Succession: The process during which one follows another in a leadership role. Succession Planning: The process that district leadership uses in preparation for filling future leadership positions. The process includes recruitment, selection, development and support of school principals. Superintendent’s Cabinet: Senior level leadership at the district level including the Superintendent and Assistant Superintendents. Tapping: The act of identifying and encouraging teachers to pursue school leadership by building principals Turnover: The rate at which the district must replace employees based on vacancies due to employee resignations. Organization of the Program Evaluation Study This problem of practice program evaluation study is divided into five chapters. Chapter One includes an introduction, a discussion of the complexity of leadership, the statement of the problem, the significance of the study, the purpose of the study, research questions addressed in 30 the study, a description of the school system, the implementation of the GROW program, problem of practice research, definitions of terms, and an overview of the study. Chapter Two includes a review of the literature relative to teacher leadership development programs along with evaluation measures used to assess program outcomes and drive future improvement initiatives. The chapter begins with the driving issues behind the problem of practice in this study, the shrinking pool of principal candidates. Much of the focus of this chapter includes research regarding designing a Grow Your Own leadership program. The chapter concludes with program evaluation methods with emphasis on Kirkpatrick's Four- Level Model of Training Evaluation, which is the evaluation model used in this study. Chapter Three contains the research methods used in this problem of practice study. In addition, the chapter includes the design of the study, selection of study participants, the program evaluation model, data collection process, research questions, and data analysis procedures. Chapter Four includes the program evaluation findings including an analysis of the findings to determine if the Grow Your Own leadership program implemented in Craven County Schools was effective. Chapter Five contains the summary of the problem of practice program evaluation conducted in this study. In addition this chapter includes study conclusions, recommendations for programmatic improvement, and recommendations for further study. CHAPTER TWO: LITERATURE REVIEW Introduction This chapter presents a review of the literature in several areas that supports the evaluation of a district Grow Your Own leadership program. As a foundation for this study, a review of the literature is necessary to determine the need for teacher leadership development programs and how evaluation measures can be used to assess program outcomes and to drive program improvements. The review will address the purpose for a school system to design a leadership program, as well as explore research relative to the process and components necessary in the development and implementation of a successful Grow Your Own teacher leadership program. This research is important to the evaluator in order to understand research-based program components. In addition, there will be a review of research that suggests an appropriate model to use for program evaluation. Program evaluation research will be used in this study to determine the effectiveness of the design, implementation, and success of a Grow Your Own teacher leadership development program implemented in Craven County Schools. The evaluation will provide a standard of assessment that will be used to both evaluate program success as well as make recommendations regarding design changes necessary to improve future program initiatives and outcomes. A summary of the research finding can be found at the end of this chapter. The Shrinking Pool of Principal Candidates As indicated in chapter one, there is a shrinking pool of qualified principal candidates (Grubb & Flessa, 2006; Guterman, 2007; Whitaker, 2001). The new roles required by school administrators along with the demands of the job are deterrents to potential leadership candidates (Winter & Morgenthal, 2002). The literature supports the responsibility of school leaders to 32 serve as visionaries, change agents, instructional leaders, curriculum experts, accountants, facility managers, and community heroes (Davis et al., 2005). Research concurs that accountability pressures, high levels of responsibility, no tenure, and poor compensation are the major factors that have a negative impact on those interested in school leadership positions (Whitaker, 2001). The changing role of school administrators has served as a deterrent to those who previously may have considered serving as a school administrator (Winter & Morgenthal, 2002). The stressful conditions under which current principals work have intensified the problem of finding quality candidates to fill principal vacancies (Whitaker, 2001). While there are adequate numbers enrolled in university programs and earning administrative degrees, few are applying for vacant positions in school leadership (Mezzacappa, 2008). A review of literature on the subject indicates that there is not so much a shortage of those who have proper licensure and certification for the position, but a lack of interest from teachers to move from the classroom to the principalship (Grubb & Flessa, 2006). Much of the research indicates that negative views of school leadership along with pressures from both the public and the media serve as de-motivators thus veteran teachers who have experiences that would prove beneficial in school leadership are refusing to consider becoming a school principal (Cranston, 2007; Harris, 2007; Winter & Morgenthal, 2002). Michael Copland (2001), in the article The Myth of the Superprincipal, states, Two decades into the current age of school reform, one can argue that we have reached the point where aggregate expectations for the principalship are so exorbitant that they exceed the limits of what might reasonably be expected from one person. (p. 4) Due to these demands, Copland suggests teachers no longer aspire to become principals (Copeland, 2001). Teachers once provided a direct pipeline to the principalship, but that is no 33 longer the case. Historically, teachers provided the pool from which administrators were selected, but current data indicates fewer and fewer teachers are seeking positions in administration (Howley, Andrianaivo, & Perry, 2005). Teachers, more than any other group, recognize the challenges of school leadership, so they are more reluctant to consider the shift from teacher to leader (Walker & Qian, 2006). In fact, one study suggests that fewer than half of teachers who hold administrator certification are willing to contemplate becoming a school leader (Cusick, 2003). The lack of interest from teachers in administration, along with the reality that many seeking a degree in administration have no desire to lead a school, exacerbates the challenge of school districts to implement successful succession plans relative to the replacement of school principals (Cooley & Shen, 1999). These statistics support principal recruitment as a critical issue that school districts must address (Winter & Morgenthal, 2002). Many districts are considering Grow Your Own plans in order to increase the pool of qualified and interested candidates for school leadership positions. Due to the change in the role of school leaders, development programs must keep pace and be innovative in preparing teachers as leaders. It is more than just having the certification; it is instilling the desire in teachers to lead. In a research article, Ann Lauder (2000), Coordinator of Leadership Assessment and Development at the Center for Excellence in Leadership, wrote, Districts are finding it progressively difficult to persuade teacher leaders to consider or prepare for the principalship. Many teacher leaders have one or more advanced degrees and are reluctant to pursue yet another traditional degree. Disappointment in traditional and theory-based preparation programs, coupled with the public demand for increased expectations on the principalship, has produced a world of new and redesigned principal preparation programs. (p. 25) 34 This charge to redesign programs to entice teachers to consider leadership is one that has not been addressed by many school districts as research is sparse when attempting to discover what elements and programs have proven successful (Canavan, 2001; Lacey, 2003; Rhodes & Brundrett, 2006; Sousa, 2003; Wilson, 2009). Businesses are much more proactive in succession planning and leadership development programs. Education has much to learn from business practices and therefore, much of the research will include what successful businesses have done to develop highly engaging succession plans that can be translated to education practices. Support for Grow Your Own Leadership Programs “Leadership is not a soft skill, an optional extra for oiling the machinery of industrial relations. It is a key factor is business success---whatever your business and however you define success” (Adair, 2009, p. 2). This fact of business equally applies to education. Recruiting, training, and supporting quality candidates in preparation for school leadership positions are critical due to the shallow pool of interested candidates. One of the most recent solutions to the principal shortage is "for school districts to grow their own principals" (Keil & Czernick, 2003, p. 2). The purposes of Grow Your Own leadership academies must have a focus on both providing leadership skills and encouraging teachers to be willing to accept the role of school leader even with the current challenges (Gutmore, 2009). There must be attention given to the perception that the job is too demanding and next to impossible since participant’s perceptions are a strong predictor relative to their willingness to apply for school leadership positions (Winter & Morgenthal, 2002). This supports the need for leadership preparation programs to address both skills and feeling of self-efficacy (Winter & Morgenthal, 2002). There is a vast amount of research supporting the advantages of Grow Your Own efforts 35 in business that can be translated to education (Adair, 2009; Barner, 2006; Byham et al., 2002; Fulmer & Goldsmith, 2001; Leviss, 2011; Lewkiw & Singh, 2007; Zenger & Folkman, 2003). “Growing your own leaders makes it easier to achieve your organization’s diversity goals by ensuring it has an appropriate gender and ethnic mix of candidates in the selection pipeline” (Byham et al., 2002, p. 8). In addition, Byham et al. (2002) suggest that recruiting and selecting those already employed in an organization allows for a clear direction for training and the ability to accelerate leadership bench strength since strengths and weaknesses of participants are already identified. Grow Your Own leadership programs must include a systematic process in which professional development goals are aligned with the strategic goals of a school district in order to adequately prepare to fill positions with the right people, who have the right skills, at the right time (Carol, 2004). Districts must be more creative and intentional in an effort to identify, attract and train potential administrators from the classroom. Those encouraged to participate in a Grow Your Own initiative may have obtained a degree, but need additional supports to consider moving into leadership roles in the school district. Others may not have even considered a future in administration. Even those teachers serving as teacher leaders need a variety of experiences in preparation for school leadership positions. “Even the best leaders improve their abilities through training, assignments, and experiences that push them to develop new skills and get further training” (Byham et al., 2002, p. 13). Without a systematic approach to developing leaders with varying skill levels, skill development can be inadequate in preparing teachers for the challenges they will face as school administrators (Byham et al., 2002). Identifying, selecting, and training effective future administrators is a challenge, and the literature indicates that this will be an ongoing trend that must be addressed in new and 36 innovative ways. According to a dissertation study conducted by Gerald M. Beach (2010): The process of becoming a principal is seldom compacted into a year or two of graduate leadership studies; rather it begins much earlier when teachers as graduate students engage in professional activities with fellow teachers and principals. Teachers experiences in informal and formal leadership, both prior to and while in a leadership program will help mold their conception of the school principalship. (pp. 19-20) Research indicates the need for districts to consider developing Grow Your Own leadership programs that are customized to the unique needs of the district and those participating in the program. Leadership Development Program Design A Grow Your Own leadership development program is not meant to supplant, but supplement current university principal preparation programs. Research suggests that traditional university preparations programs should not be the only means of development for future school leaders able to confront the challenges of school leadership in the twenty-first century (Hess & Kelly, 2005). University programs tend to place more emphasis on leadership theory and provide limited opportunities apply the theory into real-world practice (Lauder, 2000). In fact, traditional programs take a holistic approach and design curriculum based more on managerial tasks rather than transformational leadership skills (Daresh, 2004; Hess & Kelly, 2005). This research supports the need for additional development opportunities for future school leaders that includes opportunities to tackle real-world problems aligned with the current expectations for school leaders. A review of the research suggests that the development of school leaders occur along a continuum that includes building a candidate pool of qualified and passionate teacher leaders (Cheney et al., 2010). 37 Understanding this research provides an opportunity for school districts to design programs aimed at bridging the gap between current principal preparation program outcomes and school district needs relative to innovative school leaders prepared to face the challenges of leading a school in the twenty-first century. According to Joseph Murphy (2006), Chairperson of the Department of Educational Leadership at Vanderbilt University, there are several concerns relative to university principal preparation programs. He found traditional principal preparation program components included: • Non-competitive entrance requirements and no system to identify top candidates with potential to be great school leaders; • Minimal academic rigor with a curriculum not aligned to current school needs; • Lack of connection to real world practice; • Lack of variety in instructional methods; • Lack of diversity in participants in the program; • Reliance on an academic rather than professional model of practice. (Fullen, 2001) This research should be considered in designing a district Grow Your Own leadership program in order to fill the gaps found in traditional principal preparation programs. Grow Your Own leadership program initiatives designed to recruit and train school leaders are not new. Research conducted by Gerald M. Beach (2010) indicated several initiatives across the country aimed at seeking and supporting school principals. Beach (2010) addressed work done in Allegheny County Schools in West Virginia to address principal salaries, Fairfax County Public Schools to create a training and internship program, and in Maryland to establish a Principals Academy. Cheney et al. (2010) shared research conducted by The Rainwater Charitable Foundation designed to explore principal development programs across the county in 38 order to determine the elements of an effective program. The foundation searched for programs that included elements supported by research as successful in growing effective school leaders. Based on the foundation’s research, Cheney et al. (2010) suggest the following program components as part of the development of a successful leadership development program: • Programs must start by developing a competency framework which outlines the skills, knowledge, and dispositions that are aligned to district priorities and strategic directions. • The program participants are selected based on strategic, proactive, and targeted recruiting strategies identified as those who have potential as future school leaders. • The program is highly selective relative to which candidates are accepted into the program. • The training is designed to be hands-on and competency-based with authentic opportunities to lead adults, make mistakes, and grow. • The program must use multiple evaluation methods in order to have the data to determine the effectiveness of the program. While all of these initiatives were designed to attract, train and retain principals, there is little research relative to the success of leadership development programs nor to support the components found to be effective in program development and implementation. Many of the programs are promising, but long-term data is not available regarding their impact on leadership development (Cheney et al., 2010). Therefore, the program evaluator of this study found it helpful to also examine the literature relative to business practices in developing Grown Your Own programs including research supporting the program components necessary in the successful preparation of the next generation of leaders. 39 In the development of leadership programs, Donald Kirkpatrick (2007) suggests ten requirements for effective training programs. Consideration of these requirements during program development will ensure that “any evaluation will show positive results” (Kirkpatrick, 2007, p. 3). The ten requirements suggested by Kirkpatrick (2007) for an effective training program include: • Base the program on the needs of the participants. • Set learning objectives. • Schedule the program at the right time. • Hold the program at the right place with the right amenities. • Invite the right people to attend. • Select effective instructors. • Use effective techniques and aids. • Accomplish the program objectives. • Satisfy the participants. • Evaluate the program. An extensive review of business practices relative to leadership development was conducted by Leskiw and Singh (2007). In this study the researcher found five key factors that are necessary for leadership development that will prepare future leaders. The key factors include conducting a needs assessment, recruiting and selecting quality participants, designing an infrastructure to support the development initiative, developing and implementing a complete system of learning aligned with system goals and directions, and implementing an effective evaluation system (Leskiw & Singh, 2007). Implementation of the six stages identified in this 40 work as best practices will facilitate the development, design, implementation and evaluation of a school district Grow Your Own leadership development program. Leadership development is critical in any organization (Lewkiw & Singh, 2007). Historically, school districts have relied on university programs to attract and train the next generation of administrators, but current shortages of qualified and competent leaders necessitate the need for school districts to take a more proactive approach to identify, recruit, and train future leaders as part of a district’s strategic plan (Murphy, 2006). A review of the current literature reveals considerable evidence that most school districts have invested little to no efforts in the development and implementation of strategic plans or programs relative to developing teacher leaders as the next generation of school leaders (Canavan, 2001; Kirkpatrick, 2007; Lacey, 2003; Rhodes & Brundrett, 2006; Sousa, 2003; Wilson, 2009). This research further supports the need to address this problem of practice through this study of the development, implementation and evaluation of the leadership program including developing a competency framework, building a candidate pool, conducting a needs assessment, implementation of program component and evaluation of program effectiveness. Competency Framework A competency is “a cluster of related knowledge, skills and attitudes that affects a major part of one’s job (a role or responsibility), that correlates with performance on the job, that can be measured against well-accepted standards, and that can be improved via training and development” (Parry, 1996, p. 50). A competency framework defines the set of skills, knowledge and dispositions that effective school leaders must possess in order to drive change and positively impact student achievement. This framework guides program development, details elements of the program and provides a process for program evaluation (Cheney et al., 41 2010). The framework is the basis on which a candidate pool is recruited, selected, and trained. The competency also serves as a tool to evaluate program effectiveness and participant growth (Cheney et al., 2010). A review of the literature relative to the development of a competency framework produced a number of models addressing competencies essential to school leadership. One of the most heavily researched-based models is the Knowledge is Power Program (KIPP) (KIPP: Visionary leadership, 2013). According to the KIPP Leadership Competency Model (2013), leadership skills can be organized into four distinct categories. Each category has specific essential competencies aligned to the category. There are key behaviors included with each competency that are action oriented proficiency statements. Figure 3 visually depicts the leadership skills recommended in the KIPP Leadership Competency Model. Another model of a competency framework is referred to as the Iceberg Model (Spencer & Spencer, 1993). This model, developed by Lyle and Signe Spencer (1993), suggests that leadership competencies are more difficult to detect and train because they go deeper than just assessable skills. The suggestion is that qualifications and skills are only twenty percent of what drives leadership success while eighty percent are below the surface, yet highly influence leadership behavior (Spencer & Spencer, 1993). This research poses a particular challenge when developing a competency framework for leadership development programs as most of these behaviors are not easily detected or evaluated. Spencer and Spencer (1993) suggest that training must include two distinct competencies, achievement and impact of influence. Achievement is defined as “the drive and actions to set challenging goals and reach a high standard of Performance” (Spencer & Spencer, 1993). Impact of influence is “acting with the purpose of affecting the perceptions, thinking and actions of others” (Spencer & Spencer, 1993). Figure 4 42 Note. (KIPP: Visionary leadership, 2013). Figure 3. KIPP Leadership Competency Model. 43 Note. (Adapted from “The Iceberg Model” in Spencer & Spencer, Competence at Work, p. 11). Reprinted with Permission. Figure 4. Observable characteristics versus underlying competencies. 44 illustrates the types of competencies necessary for effective leadership including above the surface and below the surface skills. This research is critical in designing a leadership program that addresses all skills and competency-based practices, even though difficult to measure. Kathy Graham-Leviss (2011) suggests competencies are contingent on a candidate’s inherent talents, acquired learning, and behaviors. Inherent talents are those attributes that leaders naturally possess including innate abilities and personal characteristics, while acquired talents are those skills developed through experiences (Leviss, 2011). Behaviors are based on a combination of inherent talents and acquired learning (Leviss, 2011). This research is helpful in the development of a leadership program to include opportunities for new learning that are demonstrated using the inherent talents of the participants. This training model will ensure that behaviors are based on both innate abilities and learning, providing a more individualized approach to leadership development. New Leaders for New Schools developed the Urban Excellence Framework to address leadership characteristics that drive school improvement and student achievement (New Leaders, 2011). The findings and recommendations relative to the framework were based on the study of over one hundred schools identified as making dramatic gains under effective leaders (New Leaders, 2011). There are five categories with underlying levers that the research suggests are necessary in school leaders to promote school improvement (New Leaders, 2011). The framework suggests that principal development must teach school leaders how to pull the levers effectively (New Leaders, 2011). The study suggests using this framework to design leadership development program goals and as a measure to determine a candidate pool (New Leaders, 2011). Through this study, New Leaders for New Schools designed a framework that can be used by school districts to recruit, train, and support school leaders. The competency lists the 45 following five categories of practices that drive school improvement: • Learning and teaching • Culture • Aligned Staff • Operations and Systems • Personal Leadership (New Leaders, 2011). As noted in Figure 5, the foundation of this framework is the personal leadership modeled by the principal who sets the tone for all student and adult relationships and practices in the school. A review of the literature is clear in supporting the need for a competency framework to align training with system directions and leadership needs. A review of the research suggests that a competency framework be developed based on the core values, strategic directions and beliefs of the school district and should guide all parts of a development program from selecting candidates to the development of aspiring leaders (Cheney et al., 2010). The framework should be a critical component of the program from candidate selection, curriculum development, and as an evaluation tool (Cheney et al., 2010; Leviss, 2011; Spencer & Spencer, 1993). The competency framework should serve as the link between all elements of the program and should focus on the competencies the leadership program can develop considering the time frame of the program as well as the resources in place to support program implementation (Cheney et al., 2010). The review of the literature on the development of a competency framework will be essential in the development of a leadership development program aligned with the competencies deemed most important to an individual school district and to ensure program cohesion. The creation of this framework will help support a leadership pipeline by identifying those with the potential to develop the competencies based on the established framework. 46 Note. From New Leaders “Urban Excellence Framework”. Figure 5. Diagram of competency framework. 47 Building a Candidate Pool “The development of an acceleration pool is a critical component to the development of a Grow Your Own program. Rather than one or two targeted individuals, a pool provides a large number of candidates for leadership positions” (Byham et al., 2002). In addition, Byham et al. (2002) state, There are more good people than you think in your organization who have the necessary skills and who know the organization, its operations, and its history. The trick is to find them. Shoring up your identification system is the first step, followed by an improved succession management system to develop your talent. (p. 14) Specific strategies relative to recruitment efforts in educational leadership development are not prevalent in the literature, but targeted succession planning in the business sector is plentiful (Leskiw & Singh, 2007). A review of the literature regarding developing leaders in the business sector suggests that a leadership develop program use a rigorous selection process aligned to the competency framework to ensure candidates demonstrate the knowledge, skills and dispositions required to develop into effective school leaders (Cheney et al., 2010). In order to meet the high demands placed on public schools and school leaders, there must be targeted recruitment strategies developed to attract candidates and cultivate their interest in becoming school principals (Fullan, 2001). While traditionally overlooked, recruiting strong candidates to participate in the program is a critical step in the effort to build school leaders (Cheney et al., 2010). Cheney et al. (2010) suggest the following strategic recruitment strategies in order to select candidates that will have the greatest chance to greatest chance of becoming an effective school leader: 48 • Candidates should be high-quality and have a recommendation from their current principal or supervisor. • Candidates must be recognized as leaders within the school and have the ability to develop the capacity of others. • Recruitment must be transparent and based on the competency framework to ensure candidates possess the skills, talents, and dispositions required of program participants. • Recruitment efforts should generate a diverse pool of potential candidates. The candidate pool should have a broad representation relative to race, gender, school level, and experiences. • The selection process should be competitive. • Specific strategies can include flyers, email blasts, and website announcements. • Consideration is given to principal input, work experience, and leadership roles. • Some potential candidates may not readily recognize their leadership potential or have negative perceptions of leadership. Therefore, principals must be encouraged to identify and tap those teachers that have leadership potential. A review of the literature relative to developing a candidate pool provided a foundation on which a leadership development program cohort should be developed. “A more selective process for choosing candidates for training is an essential first step to create a more capable and diverse corps of future leaders” (Wallace, 2011). The goal of any leadership development program should be to not only prepare more school leaders, but the right ones (Wallace, 2011). Primary to any successful program is the quality of candidates selected to participate in the development program. Byham et al. (2002) suggest the following must have criterion: 49 • Minimum educational requirement • Minimum time with the organization • Required leadership experience • Specific levels on performance appraisal ratings • Sporadic training, experiences, and skills • Modeling of organizational values While this criterion is based on business leadership development, they are all applicable to school district leadership development. Using research based selection criteria will ensure those selected have leadership promise (Byham et al., 2002). In addition to building a quality candidate pool, school districts should seek to expand the ethnic and gender diversity of the participants. Stanford’s researchers found that participants in district initiated Grow Your Own leadership development program are more likely to be women (73% versus 48%) and members of minority groups (37% versus 8%) than those in traditional university programs (Davis et al., 2005). According to Browne-Ferrigno and Muth (2004), the most effective way to recruit minority leaders is through the administrators that hired them. “Administrators should see every new hire as a potential future teacher leader, and possibly a future principal” (Browne-Ferrigno &Muth, 2004, p. 476). Principals should seek to encourage minority teachers to apply for district leadership programs. Research states, “Principals who courted and relentlessly recruited their protιgιs provided the greatest influence on the teachers' decisions to become principals” (Browne-Ferrigno & Muth, 2004, p. 476). The literature supports the role of current administrators in helping the school district in the development of a candidate pool. 50 Needs Assessment A needs assessment is necessary as the underpinning of any leadership development program in order to ensure the program meets the needs of both the participant and those of the organization (Fulmer & Goldsmith, 2001). When developing a Grow Your Own leadership program, needs and competency gaps must be identified (Kesler, 2002). “Succession planning needs to be refocused away from replacement planning to include a more comprehensive set of assessment and development practices that support the entire pipeline or flow of talent” (Kesler, 2002, p. 32). A needs assessment will help define program objectives and customize training based on the unique needs of the participants and district priorities. Zenger and Folkman (2003) conducted research that supports the alignment of district priorities to leadership develop initiatives to define the desired outcomes of internal leadership development programs. A needs assessment will ensure organizations develop the bench strength of employees aligned to future needs (Barner, 2006). Program Components Primary to the development of future school leaders is the components of Grow Your Own programs. The components of the program must be rigorous and require participants to apply theory to real-world practice (Wallace, 2012). The program should include case studies, problem-solving activities, and reflection (Wallace, 2012). The literature supports having curriculum aligned to real work in order to support leadership development (Beach, 2010; Bumphus & Royal, 2008; Campbell, 2002; Cheney et al., 2010). A review of the literature suggests that the content of the leadership program align with the competency framework and provide participants and opportunity to put theory into practice (Campbell, 2002; Cheney et al., 51 2010; Wallace, 2012). According to Davis et al. (2005), research suggests the following program features: • Focus on the core values of leadership; • Curriculum addresses instructional leadership, organizational development, and change management; • Real-world experiences paired with classroom learning; • Collaboration opportunities through the development of cohort groups; • Trainers who have a strong and effective leadership experiences. Wallace (2012) contends that there must be tight alignment between the content, competency framework, and program objectives. To meet these standards, research suggests that all resources are published to the district website, sessions are debriefed, participants are surveyed, and changes are made in content and delivery based on survey results (Wallace, 2012). In addition, Zenger and Folkman (2003) suggest that participants give ongoing feedback regarding the components of the program to ensure the scope and sequence of the program aligns to participant needs and district expectations. Evaluation of Program Effectiveness Leskiw and Singh (2007) found that successful Grow Your Own programs have embedded evaluation practices to assess leadership program outcomes. Based on the research of Russon & Reinelt (2004) relative to Kellogg’s leadership programs, multiple evaluation methods provide the best data and include both qualitative and quantitative data. Surveys followed up by individual interviews provide multiple sources of data and yield a fuller picture of the impact of the leadership program (Russon & Reinelt, 2004). Regardless of the form of evaluation, the assessment must give information about 52 program impact. According to Darling-Hammond, LaPointe, Meyerson, and Cohen (2007), program effectiveness should be judged by the change in leadership practices as well as how the participants view themselves in the role of school leader. Using the competency framework as a basis for the evaluation will provide benchmark for program outcomes. Assessment should include pre and post evaluations as well as formative assessment throughout the leadership development program (Darling-Hammond, LaPointe, Meyerson, & Cohen, 2007). In addition, there should follow up with participants to determine long-term impact of the program on leadership (Bumphus & Royal, 2008). The research of Leskiw and Singh (2007) relative to meaningful evaluation of leadership development programs suggest the use of Kirkpatrick’s four-level training model as a “prominent method used to evaluate the extent to which learning takes place and it can be very useful in the evaluation of leadership development initiatives" (p. 458). Donald Kirkpatrick’s Four-Level Model of Evaluation Donald Kirkpatrick’s four-level model of training evaluation has been used to evaluate a number of trainings in a variety of settings for over forty years (Kirkpatrick, 2006). It is one of the few models used to evaluate programs developed and implemented in the education sector (Leskiw & Singh, 2007). The model was originally developed in 1952 during Kirkpatrick’s own dissertation research which involved the evaluation of training programs (Kirkpatrick, 2006). Lack of adequate evaluation models during the study led Kirkpatrick to formulate the four levels of evaluation in order to “clarify the elusive term evaluation” (Kirkpatrick, 2006, p. xv). In order to assess training based on outcomes, Kirkpatrick (2006) suggests the following sequential evaluation levels: 53 • Level 1: Reaction • Level 2: Learning • Level 3: Behavior • Level 4: Results Figure 5 illustrates the four levels and the purpose of each level in program evaluation. Donald Kirkpatrick (2006) found that the levels are interdependent, one leading to the next. The research suggests that positive reactions from program participants contributes to learning, learning impacts the desire of the participants to change behavior, and that changes in behavior leads to organizational results (Kirkpatrick, 2006). The model is depicted in Figure 6. The first level of evaluation is reaction. Reaction in related to how the participants respond to the training based on their level of satisfaction (Kirkpatrick, 2006). This level evaluates the participants’ feelings about program components, the trainer, the facility and overall satisfaction with the development program (Kirkpatrick, 2007). Evaluation at this level provides information regarding the participants’ interest and attitudes, which Kirkpatrick (2006) suggests program success is dependent. “If participants do not react favorably, the will not be motivated to learn” (Kirkpatrick, 2006, p. 22). The second level of Kirkpatrick’s model evaluates participant learning. “Learning can be defined as the extent to which participants change attitudes, improve knowledge, and/or increase skill as a result of attending the program” (Kirkpatrick, 2006, p. 22). Kirkpatrick (2006) suggests that programs designed to develop leaders should address all three areas. Kirkpatrick (2006) proposes that in order for learning to take place, one of the follow must occur: Attitudes change. There is an increase in knowledge. A skill is improved. These areas must learning must be considered when developing and evaluating a leadership development program. 54 Note. (Kirkpatrick, 2006). Figure 6. Kirkpatrick’s Model of Program Evaluation. 55 Kirkpatrick’s third level of evaluation is behavior. Kirkpatrick (2006) defines behavior as “the extent to which change in behavior has occurred because the participant attended the training program” (p. 22). Based on Kirkpatrick’s (2006) research, unless the new knowledge and skills acquired in a training program transfer to on the job behaviors, there is little to no value to the organization. In fact, Kirkpatrick (2007) found that the evaluation itself “acts as a reinforcer to new behaviors” (p. 82). The fourth level in Kirkpatrick’s model is results. Kirkpatrick (2006) defines results as “the final results that occurred because the participants attended the program” (p. 25). Kirkpatrick (2006) admits that evaluating leadership outcomes may prove difficult, yet program outcomes should lead to tangible results. Results of a leadership development program may be immediate and/or long-term, requiring formative, summative and ongoing assessment of the impact of the program, as well as follow-up with program participants in future organizational positions. Thus, the value of the program must be evaluated in term of short and long term impact. Program evaluation is essential to determine program effectiveness (Brousselle & Champagne, 2011). The model selected for this study is Kirkpatrick's Model for Program Evaluation. This model has been used for over forty years and one of the few models used to evaluate success of leadership programs in the education sector (Leskiw & Singh, 2007). The model uses four steps or levels, which align with the needs of this study relative to developing and evaluating a Grow Your Own leadership development program in Craven County Schools. Using the Kirkpatrick levels for evaluating the leadership development program would provide a sequential tool useful in building a “compelling chain of evidence as to the value of learning” to the organization (Kirkpatrick, 2007, p. 123). Figure 7 illustrates the interconnectedness of the 56 Note. (Chain of Evidence, 2014). Figure 7. Diagram of Donald Kirkpatrick’s Chain of Evidence. 57 evaluation cycle and how each connects to show a chain of evidence. This evaluation model allows for the inclusion of both quantitative and qualitative data. In addition, this model would provide comprehensive evidence helpful when presenting results to district leaders, community member, funding sources and Board of Education members Summary and Conclusions The research relative to Grow Your Own school leadership programs in education is sporadic. For that reason research from the fields of education and business were reviewed as a foundation for this study. The review revealed data on the shrinking pool of candidates, supporting the need for the development of school district initiated Grow Your Own leadership development program. Research regarding the components of the program was also reviewed including program design and elements, developing a competency framework, building a candidate pool, conducting a needs assessment, program components, and evaluation of program effectiveness. The literature review will provide the foundational research used in the development and evaluation of a grow your own leadership program in a local school district. Research findings will help guide the process of the development of a leadership program for Craven County Schools. The program will be research-based and aligned with the findings in the literature review. Bumphus and Royal (2008) provided the following summary of research that will guide the program development process: • Decide on and stick to a program mission. • Consult with an external advisor before you begin the development process. • Ongoing engagement of the participants after the formal program ends is key to successful professional development. • Choose an internal evangelist to lead the development and implementation. 58 • The program must have a foundation based on the competencies of successful leaders and include the components of successful leadership programs. • Assessment is ongoing and essential in the continuous improvement of the people, the program and the organization. These research-based program recommendations will be used to develop, implement and evaluate a leadership program designed to meet current and future leadership needs of the Craven County School System. Research was clear regarding the necessity of program evaluation, and therefore, will be an essential element in this study of the development and implementation of a district Grow Your Own leadership program. After reviewing several evaluation models, Kirkpatrick's Model for Program Evaluation was selected as the tool that will be used to assess program outcomes due to the action-oriented design of the model and its ability to provide a chain of evidence to support the outcomes of this program evaluation study. The research questions were designed to address one of the four levels of evaluation of Kirkpatrick’s model, which included reaction, learning, behavior, and results (Kirkpatrick, 2006.) Further discussion regarding the evaluation methodology and the phases of the program evaluation will be presented in Chapter Three of this study. CHAPTER THREE: METHODOLOGY Introduction The purpose of this study is to examine and evaluate one school district’s efforts to design and implement a Grow Your Own leadership development program to address the issue of a shortage of qualified candidates interested in leadership positions. This chapter presents the methods that will be used in the collection and analysis of data in order to answer the research questions from Chapter One of this study. Specifically, the study seeks to determine if those leaders identified, recruited, prepared, trained and supported through the implementation of research-based practices, are better prepared to take leadership positions in the schools than those who did not participate in the program. The program evaluator is also interested in determining if the completers have an increased level of interest in securing a school leadership position. Design and implementation effectiveness will be determined by using Kirkpatrick’s (2006) Model for Program Evaluation. This study utilizes a mixed methodology design that includes quantitative and qualitative data to evaluate the critical elements necessary in creating effective leadership development programs and to determine the success of the implementation of those components in a Grow Your Own school district leadership program. The evaluation component is based on Donald Kirkpatrick's (2006) model of training evaluation. Kirkpatrick's model is comprised of four levels of evaluation criterion: (1) reaction of program participants, (2) learning acquired by program participants, (3) behavior changes in program participants, and (4) overall results of the program on the participants and the school district. Included in this chapter is a description of the evaluation methods of the GROW Program, the design of the study, selection of study participants, discussion of the program 60 evaluation model, data collection process, review of the research questions, and the data analysis procedures to be used in the study. The chapter ends with a summary of data that will be collected and the collection procedures. Evaluation Methods of the Grow Program Based on the research of Russon and Reinelt (2004), multiple evaluation methods provide the best data and include both qualitative and quantitative data. Multiple sources of data allow for a more complete picture of the impact of the leadership program (Russon & Reinelt, 2004). In addition, Zenger and Folkman (2003) suggest that participants give ongoing feedback regarding the components of the program to ensure the scope and sequence of the program aligns to participant needs and district expectations. Based on these research findings, the following evaluation methods will be used to assess the GROW leadership development program: • Monthly participant assessment of program components • Summative evaluation through participant questionnaire • GROW evaluation, rating pre and post assessment of the understanding of leadership principles • Interviews with selected participants to gain a deeper understanding of the short and potential long-term benefits of the program For the purposes of this study, the use of Kirkpatrick’s (2006) four-level training model will be used to evaluate the impact of the program based on a review of qualitative and quantitative data sources from multiple evaluation methods. Data will be disaggregated based on the following sequential evaluation levels in Kirkpatrick’s (2006) model: • Level 1: Reaction • Level 2: Learning 61 • Level 3: Behavior • Level 4: Results Design of the Evaluation Study This study is designed to collect and analyze data relative to the success of the implementation of a Grow Your Own leadership development program in Craven County Schools. The study utilizes a mixed methodology design that includes both quantitative and qualitative research techniques in order to gain data from multiple sources. Trammel (2005) states, “mixture of data collection methods is necessary for practical reasons as well as for philosophical reasons – no single type of evaluative activity completely captures the essence of why and how” (p. 35). The quantitative data will be derived from surveys from monthly session evaluations and a GROW program completers’ survey. Surveys allow for analysis of attitudes and behaviors and “constitute one of the most important data collection tools available in evaluation" (Fitzpatrick, Sanders, & Worthen, 2004, p. 341). In addition to determining program effectiveness, the results will also be used to improve future program initiatives. Qualitative data will be derived from personal interviews of program participants. According to Gall, Gall, and Borg (2003), qualitative data is helpful in research to discover overarching themes, patterns and relationships in sample participants. The researcher intends to evaluate the GROW program using a holistic approach including participant reactions, acquired learning, application of learning, and changes based on participation in the program. The evaluation conceptual framework and data collection is based on Donald Kirkpatrick's (2006) model of training evaluation, which directly aligns to the research goals of this study. Aligned to the research of Donald Kirkpatrick, the model to be used in this study includes: • Level 1: Reaction - How did the learners react to the GROW program? 62 • Level 2: Learning - To what extent did the GROW participants change attitudes, improve knowledge, and/or increase skill level? • Level 3: Behavior – What changes in behavior occurred in GROW program participants? • Level 4: Results – What are the results of the learning acquired in the GROW program to the individual and the organization? Selection of Evaluation Study Participants The participants in this study are all completers of the GROW program implemented in Craven County Schools. For the purpose of this study, the program evaluator will survey the twenty-four participants that complete the GROW program. Surveys include monthly session evaluations and a final GROW program completers’ questionnaire. In addition, participants will complete an evaluation assessing pre and post knowledge level of core leadership principles. Finally, individual interviews will be conducted with ten program participants who volunteer to participate in this portion of the study. Interviews will be used in order to gain a deeper understanding of the program impact on individual participants and to determine the effectiveness of the program based on program goals. Prior to data collection from the participants, the researcher will submit an evaluation plan to the institutional review board (IRB) for approval. The evaluation plan will support the study having a low risk to the subjects involved in this research. Participants in the research study will be provided information regarding the collection process and how the results of the study will be used. Of the four phases of the evaluation process, three allow for anonymity. For the one phase in which participants’ identity will be identifiable, only participants that volunteer will be included in this phase of the study. All research participants will be given the right to 63 refuse to participate in any part of the study. Informed consent will be obtained from participants involved in the study in order to use surveys, participant interviews and any video clips produced during program implementation (see Appendix J). Program Evaluation Model The program evaluation model that will be used for evaluating the effectiveness of the Grow Your Own leadership program designed and implemented in Craven County Schools will be based on Donald Kirkpatrick's (2006) four-level training evaluation model. This model will address program effectiveness on four levels of evaluation in order to determine program impact and effectiveness. Level one, reaction, will determine how participants responded to the program in terms of overall satisfaction. Level two, learning, will be designed to determine if the participants acquired the expected knowledge and skills. Level three, behavior, will determine the impact the training had on job performance. Level four, results, will be designed to determine the overall impact on participants and the school district. The researcher selected Kirkpatrick’s (2006) model due to extensive research on this model as presented in Chapter II and its relevance to the desired outcome of this study. This model is appropriate for the purpose of this study to evaluate multiple outcomes relative to the design and implementation of a Grown Your Own leadership program in Craven County Schools. Kirkpatrick’s (2006) model addresses evaluation as both a formative and summative process with the intent of using multiple measures to provide rich data to determine both program effectiveness and to drive program improvement. Data Collection Process The data collection process in this study utilizes a mixed methodology design that includes quantitative and qualitative data sources that will be collected in four phases. Using 64 multiple evaluation methods will provide a more complete picture of the short and long term impacts of the implementation of the leadership development program in Craven County Schools (Russon & Reinelt, 2004). Each phase of the data collection process will be designed to give specific information required in Donald Kirkpatrick's (2006) four-level training evaluation model. Questions will be aligned to each level as explained later in this study. The study will be conducted in the following stages. Phase 1: Monthly Assessment (Quantitative and Qualitative) At the end of each GROW session, participants will be asked to evaluate the session relative to its objectives, activities, program sequence, and materials. Quantitative and qualitative data will be derived from each monthly assessment. The monthly assessment consists of six close-ended questions and an optional comment section (see Appendix K). Questions are designed to elicit a yes/no response. This fixed response method provides quantitative data that will be useful in ongoing program improvement. Qualitative data will be derived from the participant narrative comments, which will be an optional section on each session evaluation form. Using both methods allows for quick responses and in-depth comments helpful in this study and may help to increase the response rate due to the reduced time required to complete each monthly assessment. Many of the questions on the monthly assessment align to Kirkpatrick’s (2006) reaction level, which measures participant satisfaction. According to Kirkpatrick, participant satisfaction is necessary for higher-level evaluation results. The information will also be helpful in designing improvement initiatives prior to implementing future GROW programs. Phase 2: Summative Questionnaire (Quantitative and Qualitative) Phase 2 will consist of questions created by the program evaluator to be completed by 65 participants at the conclusion of the GROW program (see Appendix L). Data will be collected based on eight questions to include a combination of rating-scale items and open-ended questions, which will provide the opportunity for individual participant comments. Data collected will be both quantitative and qualitative in nature. Quantitative data will be derived from a three level Likert-type scale indicating agreement with statements using the terms More Likely, Less Likely and No Change. In addition there will be several Yes/No response questions. In order to gain a deeper understanding of the impact of the program, participants will be asked to elaborate on each yes/no response with further details addressing the “why” relative to each response. Phase 3: GROW Evaluation (Quantitative) At the conclusion of the GROW program, each participant will complete a retrospective post-then-pre evaluation relative to individual growth based on program participation (see Appendix M). This form of quantitative data will be gathered in order to assess program impact using a 7-point Likert-type scale. Each participant will be assigned a number in order for responses to be anonymous, which may help support the validity of responses. A retrospective post-then-pre assessment will reduce the possibility of response shift bias and will present individual participant data in a way that remains anonymous. Analysis of the GROW evaluation data will provide information for program improvement in the areas of content, format and learning activities. Phase 4: Participant Interviews (Qualitative) Phase 4 of the data collection process will be a semi-structured interview protocol in order for the program evaluator to develop contextual data relative to program impact and the influence of the program on individual participants (see Appendix N). Data collected will 66 include demographics about program participants (gender, age, education level, and years of experience in education) as well as data that will be used to determine how the program impacted participants and their interest in becoming school leaders. Ten program participants will be interviewed using a set of open-ended questions allowing the program evaluator to discover any other data patterns relative to program outcomes that will likely not be revealed in other data sources. The interviews will be conducted six months after the completion of the GROW program in order to determine if transfer of learning occurred and to allow time for participants to apply for school leadership positions. The interview protocol will be used to discover patterns of experiences aligned to program impact on participants as well as to generate input that will be used to inform improvement initiatives in future program implementations. Interviews will be taped and later transcribed in order to organize data in the appropriate evaluation level of Kirkpatrick’s (2006) model of program evaluation. This process will allow the program evaluator to determine if trends exist relative to program impact on individual participants. Analysis of interview protocol data will provide evidence of on-going use of program learning and will be used to restructure future leadership programming. Research Questions After a thorough review of the literature as disclosed in Chapter Two, the program evaluator worked with an outside consultant to develop and implement a leadership development program in Craven County Schools. The GROW program was developed and implemented based on current district needs using research based best practices. A critical component of any district Grow Your Own leadership initiative is to evaluate the program, which is the purpose of this study. The study is a program evaluation designed to evaluate the success of the program using Kirkpatrick’s (2006) model of program evaluation. A review of this evaluation model was 67 used to develop a list of questions to be used in quantitative and qualitative research techniques in order to gain multiple data sources to evaluate program outcomes. This study is divided into four phases of evaluation with questions aligned to Kirkpatrick’s (2006) model of evaluation. The phases of evaluation will include (1) Monthly participant assessment of program components, (2) A summative questionnaire, (3) Evaluation of learning through a pre and post assessment of understanding of core leadership principles and (4) Interviews with selected participants to gain a deeper understanding of the short and long term benefits of the program. Kirkpatrick’s (2006) model was used to formulate central research questions. Under each central research question, sub-questions will be used to gather data aligned with each level of Kirkpatrick’s (2006) model. The questions for level 1 (reaction) were designed to evaluate the participants’ reaction to the GROW program. Level 2 (learning) questions were developed to evaluate participants’ learning relative to the core leadership principles addressed in the GROW program. Questions in level 3 (behavior), were designed to evaluate the changes in behavior of the participants of the GROW program. Level 4 (results) questions will be used to determine the results of the program and to determine the need for program improvements. For the purpose of the study, four sets of questions were formulated under the headings of the four levels of evaluation proposed by Kirkpatrick (2006). Level 1: Reaction 1. Did the GROW program meet the needs participants? 1.1 Were the objectives of the program sessions clear? 1.2 Did the learning activities enhance understanding of leadership concepts? 1.3 Was the sequence of the sessions appropriate? 1.4 Did the participants gain awareness of new leaderships concepts? 68 1.5 Did participants feel the information learned will assist with current and future work? 1.6 Were the session handouts helpful? 1.7 Why did participants initially apply for the GROW Program? Did someone encourage participants to apply? If so, did that impact the decision to apply? 1.8 If participants had to do it over again, would they choose to participate in the GROW program? 1.9 Did the participants feel the program was beneficial? 1.10 Was the program leader knowledgeable and helpful? 1.11 What did the participants identify as the most and least beneficial aspects of the program? Level 2: Learning 2. What did participants in the GROW program learn relative to leadership principles? 2.1 Which leadership principles were developed as a result of participating in the GROW program? 2.2 Did the GROW program help develop a stronger pool of applicants for school leadership positions? 2.3 What did participants report learning as a result of participating in the program? Level 3: Behavior 3. How has the behavior and attitudes of the participants in the GROW program changed relative to school leadership as a result of participating in the program? 69 3.1 Did program participation increase participant interest in advancing to leadership roles? 3.2 How did the program shape the beliefs of the participants? 3.3 How have participants applied learning from the GROW program to their daily work? 3.4 What aspect(s) of the program most influenced the beliefs of the participants? 3.5 What topics in the program most changed the behavior of the participants? 3.6 Has the participants’ interested in leadership changed? 3.7 Are participants more or less interested in becoming a school leader since participating in the GROW program? 3.8 What are the future plans of the participants relative to school leadership? Level 4: Results 4. Did the GROW program meet the goal of designing and implementing a program effective in increasing the pool of qualified and competent leaders in Craven County Schools? 4.1 Do participants feel the GROW program will help develop a stronger pool of applicant for school leadership positions? 4.2 What did participants identify as the most effective elements of the GROW program? 4.3 How well prepared do participants feel to become leaders in their schools? 4.4 Do participants feel there was an appropriate return on investment of the GROW Program? 70 4.5 How have participants applied what was learned in the GROW Program in their daily work? 4.6 Have the participants’ interest in leadership changed? 4.7 Have participants transitioned to a new leadership role since participating in the GROW program? 4.8 What do participants report as benefits of the program to themselves, those they lead, and to the district? 4.9 Has participant interest in leadership changed? 4.10 What additional supports would participants recommend the district put in place to further assist in leadership development in the district? 4.11 Did participants encourage others to apply for the GROW II Program? 4.12 What percentage of participants of the GROW program have transitioned to leadership positions within the school district? Each of the research questions and sub-questions are designed to align with Kirkpatrick’s (2006) model of program evaluation and to produce data to determine if the development and implementation of a Grow Your Own leadership program in Craven County Schools has produced the desired outcome of providing high quality candidates to fill anticipated leadership vacancies. The breakdown of each question, its alignment to Kirkpatrick’s evaluation level and the data source used to evaluate the question are defined in Table 2 Data Analysis Procedures Both quantitative and qualitative data methods will be used for this study and will be analyzed based on Kirkpatrick’s (2006) model of program evaluation. Data analysis will include a review of monthly assessments, summative questionnaires, GROW evaluations and transcripts Table 2 Summary of Data Collection Process Evaluation Level Primary Research Question Sub-questions Data Sources Level 1: Reaction Did the GROW program meet the needs of the participants? 1.1 Were the objectives of the program session clear? 1.2 Did the learning activities enhance understanding of leadership concepts? 1.3 Was the sequence of the sessions appropriate? 1.4 Did the participants gain awareness of new leaderships concepts? 1.5 Did participants feel the information learned will assist with current and future work? 1.6 Were the session handouts helpful? ? Monthly Assessments 1.7 Why did participants initially apply for the GROW Program? Did someone encourage participants to apply? If so, did that impact the decision to apply? 1.8 If participants had to do it over again, would they choose to participate in the GROW program? 1.9 Did the participants feel the program was beneficial? 1.10 Was the program leader knowledgeable and helpful? 1.11 What did the participants identify as the most and least beneficial aspects of the program? ? Participant Interviews 71 Table 2 (continued) Evaluation Level Primary Research Question Sub-questions Data Sources Level 2: Learning What did participants in the GROW program learn relative to leadership principles? 2.1 Which leadership principles were most developed as a results of participating in the GROW program? ? GROW Participant Pre/Post Evaluation 2.2 Did the GROW program help develop a stronger pool of applicants for school leadership positions? 2.3 What did participants report learning as a result of participating in the GROW program? ? GROW Participant Pre/Post Evaluation ? Participant Interviews Level 3: Behavior How has the behavior and attitudes of the participants in the GROW program relative to school leadership changed as a result of participating in the program? 3.1 Did program participation increase participant interest in advancing to leadership roles? 3.2 How did the program shape the beliefs of the participants? ? Summative Questionnaire 3.3 How have participants applied learning from the GROW program to their daily work?? 3.4 What aspect(s) of the program most influenced the beliefs of the participants? 3.5 What topics in the program most changed the behavior of the participants? 3.6 Has the participants’ interested in leadership changed? ? Participant Interviews 72 Table 2 (continued) Evaluation Level Primary Research Question Sub-questions Data Sources 3.7 Are participants more or less interested in becoming a school leader since participating in the GROW program? 3.8 What are the future plans of the participants relative to school leadership? Level 4: Results Did the GROW program meet the goal of designing and implementing a program effective in increasing the pool of qualified and competent leaders in Craven County Schools? 4.1 Do participants feel the GROW program will help develop a stronger pool of applicant for school leadership positions? 4.2 What did participants identify as the most effective elements of the GROW program? 4.3 How well prepared do participants feel to become leaders in their schools? 4.4 Do participants feel there was an appropriate return on investment of the GROW Program? 4.5 How have participants applied what was learned in the GROW Program in their daily work? 4.6 Have the participants’ interest in leadership changed? 4.7 Have participants transitioned to a new leadership role since participating in the GROW program? 4.8 What do participants report as benefits of the program to themselves, those they lead, and to the district? 4.9 Has participant interest in leadership changed? ? Summative Questionnaire ? Participant Interviews 73 Table 2 (continued) Evaluation Level Primary Research Question Sub-questions Data Sources 4.10 What additional supports would participants recommend the district put in place to further assist in leadership development in the district? 4.11 Did participants encourage others to apply for the GROW II Program? 4.12 What percentage of participants of the GROW program have transitioned to leadership positions within the school district for the GROW II Program? ? Review of District Employment Data compared to GROW participant list 74 75 of participant interviews. Each evaluation method will address a different level of Kirkpatrick’s (2006) model. A large part of the data will be gleaned from participant interviews as the analysis of the transcripts will address each of the four areas of Kirkpatrick’s (2006) model of program evaluation. Since much of the data to be analyzed will come from participant interviews, the participants will be given the opportunity to review the transcripts to ensure validity of the study. Data analysis will be derived from a mixed-method approach permitting multiple forms of data used to evaluate efforts to design and implement a Grow Your Own leadership development program in Craven County Schools. The program evaluator will analyze the data in order to look for patterns in responses allowing for conclusions to be drawn relative to the research questions posed in this study. In addition the research will review demographic variables to determine if the impact was consistent for all participants regardless of these variables. Variables will include gender, years in education and current position. A summary of data analysis collection methods that will be used in this study, along with the evaluation level each method will address relative to Kirkpatrick’s (2006) model of program evaluation is contained in Table 3. Summary Chapter Three outlines the methodology that will be used in this study aimed in determining the success of one school system in developing and implementing a Grow Your Own leadership development program designed to address the problem of the shortage of qualified candidates for leadership positions. The specific program to be evaluated is the GROW Program, designed and implemented in Craven County Schools based on the strategic directions and goals of the district to increase the quantity and quality of candidates for school leadership positions. 76 Table 3 Summary of Data Analysis Data Collection Method Evaluation Level 1: Reaction Evaluation Level 2: Learning Evaluation Level 3: Behavior Evaluation Level 4: Results Monthly Assessments • Summative Questionnaire • • GROW Evaluation • Participant Interviews • • • • 77 Included in this chapter is a description of the evaluation methods of the GROW Program, the design of the study, selection of study participants, discussion of the program evaluation model, data collection process, review of the research questions, and the data analysis procedures to be used in the study. The program evaluator of this study will use both qualitative and quantitative methods to assessment the GROW Program implemented in Craven County Schools. The framework used to assess the program will be based on Kirkpatrick’s (2006) model of program evaluation to include: • Level 1: Reaction - How well did the learners react to the GROW program? • Level 2: Learning - To what extent did the GROW participants change attitudes, improve knowledge, and/or increase skill level? • Level 3: Behavior – What changes in behavior occurred in GROW program participants? • Level 4: Results – What are the results of the learning acquired in the GROW program to the individual and the organization? This research was unique to the school system described in this study, but other school systems may find the study helpful in designing, implementing and evaluating efforts to deepen the pool of qualified candidates for school leadership positions within their own districts. The findings of this study will be presented in Chapter Four. CHAPTER FOUR: FINDINGS Introduction This study used an action-oriented program evaluation research model to provide a chain of evidence necessary in determining the success of the implementation of a district Grow Your Own teacher leadership program in Craven County Schools. Specifically, the program evaluation will determine if implementation of the GROW program had a positive impact on participants and addressed the problem of the lack of qualified candidates currently within the district. Patton (2008) defines program evaluation as the systematic collection of information about the activities, characteristics, and results of programs to make judgments about the program, improve or further develop program effectiveness, inform decisions about future programming, and/or increase understanding (p. 39). A mixed methodology approach was used in this study. Data was gathered in four phases and included quantitative and qualitative data sources in order to assess program quality and effectiveness. The phases of evaluation included (1) A monthly participant assessment of program components, (2) A summative questionnaire, (3) Evaluation of learning through a pre and post assessment of the level of understanding of core leadership principles and (4) Interviews with selected participants The framework for data collection was based on Kirkpatrick’s (2006) model of program evaluation to include: • Level 1: Reaction - How well did the learners react to the GROW program? • Level 2: Learning - To what extent did the GROW participants change attitudes, improve knowledge, and/or increase skill level? • Level 3: Behavior – What changes in behavior occurred in GROW program participants? 79 • Level 4: Results – What are the results of the knowledge acquired in the GROW program to the individual and the organization? Through an analysis of the data collected at each level, the study addresses the primary research question of this study: How effective is a school system’s research-based Grow Your Own leadership development program in growing a pool of qualified candidates to fill future leadership positions? In addition, data from the study will be used to identify program strengths and areas for improvement. Other uses of the results of this study will be to present data to district leadership regarding return on investment, to secure funding sources for future program initiatives that prove effective, and to share with other school districts with similar issues relative to developing a pipeline of leaders for impending shortages. Chapter IV presents the results of the analysis of data from the four phases of data collection in order to answer the research questions presented in Chapter One of this study. This chapter includes a description of the GROW program that is being evaluated, the characteristics of study participants, a description of the phases of data collection, response rate data, and findings based on Kirkpatrick’s Training Evaluation Model. The chapter concludes with a review of the impact of the GROW program by discussing emergent themes and a chapter summary. Description of the GROW Program Due to the challenge Craven County Schools faced relative to filling leadership positions within the district, the five year strategic plan for the district was revised to include a goal to create a culture that attracts, supports, and retains high-quality staff. As a key strategy and in an attempt to address the limited pool of candidates for leadership positions within the district, Craven County Schools designed and implemented a leadership development program in order to 80 support future leaders in Gaining skills, Realizing their potential, Optimizing their Impact on Others, and increasing Work Satisfaction (GROW). The GROW program was designed to address facilitating and building the leadership skills of teacher leaders as part of a multifaceted approach to succession planning for the purpose of the identification, recruitment, and development of future school leaders. The objective of the GROW program was to develop leaders from within the organization while deepening the pool of quality candidates as a future source for school administrators. The primary function of the program was to equip teacher leaders with the insights, knowledge and skills essential to sustaining systemic improvements for the short and long term organizational goals of developing effective school leaders. The GROW program was implemented in four stages. Stage one involved engaging an external consultant to aide in the program development process. Stage two included developing the program objectives and intended outcomes. Stage three developed the standards for program participants. Stage four defined the program topics. After the program was developed, participants were selected and the 10-month teacher leadership program was implemented in Craven County Schools in the fall of 2013. Twenty five candidates became part of the initial cohort class of GROW. The GROW program included ten monthly hands-on and competency-based sessions with authentic opportunities to lead adults, make mistakes, and grow. The program was designed to address the individual and collective developmental needs of designated participants as set forth in the selection criteria and in relationship to the human, conceptual and technical skills required of school leaders to sustain enhanced student performance in Craven County Schools. The leadership principles that served as a foundation for the program are as follows: 81 • Know Thyself • Core Values • Courage • Know Thy Territory • Building a Culture • Goal Setting • Communication • Decision Making • Impact Analysis • Conflict Management • Accountability The GROW program concluded in May of 2014. The data collected and analyzed in this chapter is based on evaluation data gathered from the twenty four participants that completed the GROW program. Characteristics of Study Participants The study population consisted of the twenty four completers of the initial GROW program implemented in Craven County Schools. All GROW graduates were invited to participate in at least one phase of this research study. Of the participants in the GROW program, 100% were asked to participate in the monthly assessments, summative questionnaire, and the GROW evaluation (n=24). Forty-two percent of the GROW participants were invited to participate in individual interviews (n=10). Before sharing the outcomes of the research, understanding the characteristics of the research participants is important. Table 4 describes the participants of the GROW program 82 Table 4 Demographics and Background of Study Participants Variable % N Gender Male 17 4 Female 83 20 Ethnicity Caucasian 87 21 African American 13 3 Years in Education 3 – 5 Years 29 7 6 – 10 Years 25 6 11 – 15 Years 25 6 16 + Years 20 5 Current Assignment Elementary (K-5) 54 13 Middle School (6-8) 21 5 High School (9-12) 17 4 Central Office 8 2 Level of Education Bachelor’s Degree 54 13 Master’s Degree 46 11 National Board Certification 13 3 83 based on gender, ethnicity, years of experience in education, current teaching assignments, highest degree obtained, and national board certification status. Each of the participants, as noted in Table 4, was invited to contribute data sources for at least one phase of this study, while several participants were involved in multiple phases of the study. The study population consisted of twenty-four full time employees of Craven County Schools who were selected to participate in the GROW program from the sixty-four applicants. Demographic information was collected from the application data. Eighty-three percent of the participants were female (n=20) and 17% were male (n=4). Eighty-one percent of the participants self-identified their ethnicity as Caucasian (n=21) and 13% self-identified as African American (n=3). Twenty-nine percent of participants have three to five years experience in education (n=7), 25% have six to nine years of experience (n=6), 25% have eleven to fifteen years experience (n=6), and 20% of the respondents reported more than sixteen years of experience in education (n=5). Review of current teaching assignments include 54% in elementary schools (n=13), 21% are teaching in middle schools (n=5), 17% of participants are high school teachers (n= 4), and eight participants work in a lead teacher role at central office (n=2). All participants either hold a bachelor’s or master’s degree. Fifty-four percent of participants have a bachelor’s degree (n=14) and 46% report earning a master’s degree (n=11). Thirteen percent of the participants have achieved National Board Certification for Professional Teaching Standards (n=3). Data Collection The data collection process in this study utilized a mixed methodology design that included quantitative and qualitative data sources collected in four phases. Four evaluation sources were used in order to gain a deeper understanding of the impact of the GROW program on participants as implemented in Craven County Schools. Monthly assessments and a GROW 84 evaluation provided quantitative data, while a summative questionnaire was used to gain both quantitative and qualitative data for this study. Ten participants were interviewed to provide additional qualitative data and to determine key points and overarching themes useful in analyzing study results. Each phase of the data collection was designed to give information aligned with Donald Kirkpatrick's (2006) four-level training evaluation model. The study's protocols were approved by the Institutional Review Board on January 5, 2015 (see Appendix O). A description of the collection of the data from each source is detailed below. Monthly Assessments Data collected from the monthly assessments included a review of all completed assessments from each of the ten GROW session (see Appendix L). A review of participant responses on the assessments included data relative to program objectives, activities, program sequence, and materials. The close-ended survey required a yes/no response to six statements relative to participants’ reactions to the training. Close-ended responses allow the program evaluator to analyze the data using a uniform method and report the data using numbers and percentages. In addition to the quantitative data derived from the monthly assessments, qualitative data was derived from the optional comment section on the survey in which participants could provide narrative comments. Assessments were completed on the following dates: • October 14, 2013 • November 8, 2013 • December 11, 2013 • February 17, 2014 • March 17, 2014 85 • April 25, 2014 • May 10, 2014 • June 10, 2014 The data was reviewed in February 2015 as a part of the research study. Summative Questionnaire A questionnaire was used to gather summative data regarding the impact of the GROW program on participants (see Appendix M). Questionnaires were given to all completers of the GROW program on June 10, 2014. Program participants were directed to return the completed questionnaires in a sealed envelope with no identifying information. The summative questionnaire included a combination of rating-scale items and open-ended questions regarding the overall perceptions and knowledge acquired due to participation in the GROW program. Questionnaires yielded both quantitative and qualitative data. Quantitative data was derived from a three level Likert-type scale indicating agreement with statements using the terms More Likely, Less Likely and No Change. In addition, there were several Yes/No response questions. In order to gain a deeper understanding of the impact of the program through qualitative data, participants were asked to elaborate on each response with further details. GROW Evaluation In order to gather data to determine the impact of the GROW program on leadership development, each participant completed a retrospective post-then-pre evaluation relative to individual growth in knowledge based on the leadership principles addressed in the program using a 7-point Likert-type scale (see Appendix N). Participants were asked to rate their level of knowledge from a “1” signifying little knowledge, to a “7” signifying a great deal of knowledge of the leadership principle. The evaluations were delivered to all participants on June 10, 2014, 86 the last session of the program. Responses were anonymous, therefore, participants were asked to return the evaluations in a sealed envelope with no identifying information. The analysis of the pre- and post-survey provided data on the knowledge and understanding of the eleven leadership principles addressed in the program and helped determine if the objectives of the program were achieved. Participant Interviews Following data collection and initial analysis of other data sources, ten program completers were purposively selected to participate in the individual interviews which yielded contextual data relative to impact of the program on individual participations. Nineteen program participants volunteered to be interviewed and ten were selected to be interviewed using a semi- structured interview protocol (see Appendix N). Stratified random sampling was used to ensure participation from a cross section of program participants. The demographics of those selected to be interviewed are presented in Table 5. Of those interviewed, 30% were male (n=3), and 70% were female (n=7). Eighty percent of interview participants self-identified their ethnicity as Caucasian (n=8), and 20% self-identified as African American (n=2). Twenty percent of those interviewed currently teach at the elementary level (n=2), 10% are current middle school teachers (n=1) and 20% are high school teachers (n=2). Of particular interest to the researcher were those participants who have transitioned into assistant principal positions since completing the program, therefore 50% of those interviewed are currently serving as assistant principals (n=5). All interviews were recorded and transcribed with participants’ permission. Transcriptions were entered into a spreadsheet using Microsoft Excel (2007) in order to allow for coding of text for analysis across questions and interview responses to identify themes regarding 87 Table 5 Demographics of Interview Participants Variable % N Gender Male 30 3 Female 70 7 Ethnicity Caucasian 80 8 African American 20 2 Current Assignment Elementary Teacher (K-5) 20 2 Middle School Teacher (6-8) 10 1 High School Teacher (9-12) 20 2 Assistant Principal 50 5 88 program impact and the development of leadership competencies. The interview questions were designed to align with each of Kirkpatrick’s (2006) model of program evaluation and to present the opportunity for participants to share personal impact of the program and the program evaluator to analyze data for common, recurring, and emergent themes. Interviews yield more detailed data in the words of the participants and allow for probing in order to provide personal responses that can often give life and meaning to the numbers presented in the tables and graphs (Lavinghouze, Price, & Smith, 2007). Analysis of the interview data revealed emergent themes that will be discussed in detail later in this chapter. Response Rates High response rates are important to the validity of research and lend to greater credibility when presenting research outcomes (Rogelberg & Stanton, 2007). Therefore, the program evaluator reviewed average response rate data to ensure an adequate return rate for all phases of this study. Based on national data, the average response rate for individuals that contributed data for published academic work is 52.7% (Baruch & Holtom, 2008). The target response rate was 85% or higher in each phase of data collection for this study. The data for this study was collected in four phases including monthly assessments, summative questionnaires, GROW evaluations and participant interviews. The response rates for the phases in this study ranged from 88% (n=24) to 100% (n=10). The response rate for each phase is depicted in Table 6, with an overall average response rate of 94.6% (n=303) for all data collection methods used is this study, which exceeds the target response rate set for this study. According to Mayfield (2013), research suggests that response rates increase and the quality of research improves when participants believe that their opinions will be heard and will affect an issue about which they deeply care, which numerous participants indicated as the case in this 89 Table 6 Response Rates Data Source Number Attempted Number Obtained Response Rate (%) Monthly Assessments 240 232 97 Summative Questionnaire 24 21 88 GROW Evaluation 24 21 88 Participant Interviews 10 10 100 90 evaluation study. A high response rate also helps to ensure the results are representative of the entire program population and improves the authenticity of this evaluation study. Findings Based on Kirkpatrick’s Training Evaluation Model Michael Patton (2008) suggests that analysis of data in a program evaluation follows four processes: analysis of data for patterns, interpretation of the significance of the data, judgment of the results, and recommendation for action based on research outcomes. “In the simplest terms, program evaluations are said to answer three questions: What? So What? Now what?” (Patton, 2008, p. 5). The findings of this study will follow the process suggested by Patton (2008) and will be presented based on the multi-level evaluation methodology suggested in Kirkpatrick’s Training Evaluation Model (2006). An analysis for each level in Kirkpatrick’s model is provided along with a discussion of each data source used to answer research question aligned to each level in order to build a chain of evidence used to answer the primary research questions. At the end of each evaluation level, there is a discussion relative to the themes that emerged from the analysis of the data sources used in that level. For the purpose of this study, at least 25% of participants must reference the topic or the topic must occur six times for it to be considered an emerging theme. Level 1 Evaluation: Reaction Research question. Did the GROW program meet the needs of the participants? According to Kirkpatrick (2006), there is direct alignment to effectiveness of training and the participants’ reaction to the training. The program evaluator analyzed data from multiple sources to determine how participants reacted to the GROW program. Data sources for level 1 evaluation included monthly assessments completed after each training session, a summative questionnaire completed at the end of the training, and participant interviews. 91 Monthly assessments. Reaction results were derived from surveys administered to all participants at each of the ten month sessions. There were 232 surveys returned yielding a response rate of 97%. The close-ended survey asked the participants to provide a yes/no response to the following six statements relative to the participants’ reaction to the GROW session: • The objectives of the session were clear. • The learning activities enhanced my understanding of concepts. • The sequence of the session was appropriate. • I gained awareness of some new concepts. • The information learned will assist me in my current and future work. • The session handouts were helpful. Figure 8 presents the response rate for each monthly survey along with the percent of yes/no responses. All data from the survey supported a favorable reaction of participants to the GROW program. In fact, all statements reflected a 100% positive rating by indicating a yes response. In addition to the close-ended survey responses, an optional comment section was included on each monthly survey to provide participants an opportunity to add additional anonymous remarks relative to their reactions to the GROW session. There were 94 comments added in the optional comment section over the 10 months of the training sessions. In addition to comments demonstrating a positive reaction to the program such as, “WOW. Just WOW!”, “Fabulous session!”, “Awesome as usual! Enjoy coming to our sessions.” and “Great as usual! Couldn’t wait to get here this morning!”, further analysis of the participants’ comments yielded several emergent themes. 92 Monthly Assessment Data Results 0 6 12 18 24 Session Months N um be r o f P ar tic ip an ts Yes 24 21 22 21 20 24 20 24 No 0 0 0 0 0 0 0 0 October November December February March April May June Figure 8. Bar graph showing monthly assessment data results. 93 Theme – self-reflection. Self-reflection was referenced 12 times in the comment section of the monthly surveys. Some of the participant responses related to self-reflection were as follows: • “The self-reflection today was awesome and very deep! I needed it.” • “Always interesting to see where one stands with self-evaluation and how to pick up strategies to improve. Thank you for helping all of us grow.” • “A lot of powerful information & reflection. These sessions are going to allow us to dig deeper in ways that will decrease our gaps.” • “Self-reflection processes/instruments very helpful, especially the homework assignments.” • “This program has really made me GROW as a leader and reflective practitioner.” “Today really helped me to feel very self aware and I know what skills I need to hone in order to be a more effective leader.” • “Today encouraged self-reflection more than probably ANY in-service session I have attended in the system. VERY good!” Theme-collaboration. Another theme that emerged from the analysis of comments from participants was the value of working with colleagues. Collaboration was highly valued and mentioned 11 times in survey comments. Comments regarding collaborations included: • “I really liked changing groups frequently.” • “Love all the opportunities to collaborate with all participants.” • “Enjoy all of the learning activities where we have to interact with peers from other schools.” 94 • “I feel as if these sessions are really helping us to know our colleagues and to bring a sense of unity to us as a group. Feel much more comfortable now sharing and asking questions of others.” Theme-presenters. There were eight comments made regarding a high level of satisfaction with the presenters, illustrating a positive reaction to the trainers. Comments specific to the reaction to the presenters included: • “Nice/knowledgeable presenters.” • “Thank you for the hard work the presenters put into each session.” • “Thanks to the presenters for their hard work in planning, preparing for and instructing us.” • “Presenters make me think!! Presenters give me no room to hide! Thank you!” • “We could all learn from the way a variety of teaching techniques and tools the presenters use!” • “The presenters’ novel and creative “hands on” approaches to teaching adults need to be shared with others who conduct workshops.” Theme-videos. A final theme evident in the analysis of the comments made by participants on the monthly surveys is related to a high level of satisfaction with the use of videos during GROW sessions. Based on comments spread throughout the ten-month sessions, the videos were helpful in engaging the participants and helping participants make connections with the concepts of the session. Comments made by participants related to the positive reaction related to video usage included: • “Really enjoyed the video.” • “Great as usual. Loved the videos.” 95 • “Always look forward to the motivating videos used to close our sessions.” Participant interviews. Ten GROW program participants were purposively selected to participate in the individual interviews in order to gain a more comprehensive understanding of the reaction of the participants to the program. The interview protocol contained six questions related to the reaction level in Kirkpatrick’s model (2006). The participants were asked to respond to the following: • Why did you initially apply for the GROW Program? Did someone encourage you to apply? If so, did that impact your decision to apply? • Tell me about your overall program experience in the GROW Program. • What aspect of the program was most valuable to you? • What aspect of the program was least valuable to you? • Do you feel the information learned in the GROW Program will assist with your current and future work? • If you had to do it all over again, would you participate in the GROW Program? Why or why not? The first question contained two parts. One part of the question was designed to determine if tapping, as defined by Grunow, Horng, and Loeb (2010), occurred as a possible incentive for teachers to consider leadership opportunities. Quantitative data was derived from this question as it presented the opportunity for participants to respond to a Yes/No option. Based on analysis of the responses, 80% (n=8) of participants reported tapping as a positive motivator that influenced their decision to apply for the GROW program, while 20% (n=2) reported that tapping had no influence on their decision to apply. Data collected is presented in Figure 9. 96 Number of Participants Encouraged to Apply to the GROW Program 0 2 4 6 8 10 Yes No Participant Responses N um be r o f P ar tic ip an ts Figure 9. Bar graph showing participant responses regarding tapping. 97 The additional part of interview question one was used to gather qualitative data in order to determine why program participants originally applied to the GROW program. Each of the 10 participants interviewed responded to this question. Using thematic analysis, comments were coded in order to identify common patterns and themes in participant responses. Three emerging themes were evident in the coded responses; tapping, school leadership, and teacher leadership Theme–tapping. The research of Grunow, Horng, and Loeb (2010) suggests that teachers who are tapped by principals are more motivated to consider school leadership positions and that the tapping increases the probability that the teacher will become a principal in the future. One participant that was interviewed stated, “Sometimes it takes someone else to identify things in you for you to start to see those things in yourself” (I-9). The researcher found evidence of the influence in tapping in this study as 80% of participants report being tapped by either a principal or central office personnel. In addition to principal tapping, Some participants reported that colleagues influenced their decision to apply for the GROW program. Participant statements supporting the influence of tapping by leaders and colleagues on their decision to apply for the GROW program include: • “A Director at Central Services encouraged me to apply for the grow program for the experience. Her encouraging me to apply had a large impact on me completing the application” (I-1). • “The email got me interested, but it was my current and former principal encouraging me that really made me apply” (I-3). • “A leader I respected encouraged me by sharing that I was on her leadership radar” (I-4). 98 • “I always wanted to lead, but encouragement of coworkers and administrators affirmed my interest in leadership” (I-5). • “I was approached by my administrator because he saw something in me that he thought would mesh with the GROW program” (I-9). • “Several people encouraged me, especially my principal. Those pushes influenced my decision to apply” (I-10). Theme–school leadership. Another theme discovered when analyzing participant interview data regarding initial reason for applying for the GROW program was the interest of applicants in future school leadership positions. Fifty percent (n=5) of those participants interviewed stated they had an interest in learning more about leadership in a role beyond that of a classroom teacher. Those interviewed had an interest in become a principal or future leader at Central Services. Participant comments relative to an interest in school leadership as the initial reason for apply for the GROW program included: • “I was already working on my masters at Garner Webb so it looked like a good opportunity. It looked like a good thing to compliment the leadership skills I was learning in my masters program” (I-2). • “I've always wanted to lead in the school system” (I-6). • “I was also exploring the idea of one day going into school administration” (I-7). • “I had always been a leader in the school, so this was an opportunity for me to step out and explore leadership at a deeper level” (I-9). Theme–teacher leadership. A final theme discovered when analyzing participant interview data regarding the reason for initial application for the GROW program was the desire of applicants to become more effective teacher leaders. Forty percent (n=4) of those participants 99 interviewed initially reported little to no interest in becoming principals, but their responses indicated that they wanted to learn how to be better teacher leaders. Interview responses supporting teacher leadership as the initial reason for applying for the GROW program included: • “I was already in a teacher leadership position, but had no aspiration to be an assistant principal” (I-4). • “I knew I needed to grow as a teacher leader” (I-5). • “I applied because I want to be a better leader in my classroom and be a better leader for the students” (I-7). Additional interview questions were developed to determine participant reaction to the GROW program focused on the program experiences including the value of the program to current and future work. Thematic analysis of participant responses regarding the reaction to program experiences revealed three emergent themes; self-reflection, collaboration, and interest in leadership. Theme–self-reflection. Reviewing coding patterns in the interview data, self-reflection stated by 70% (n=7) of participants when discussing their reactions relative to the positive aspects of the program. When participants shared comments about the overall program experience, the aspects of the program that were most valuable, and program elements participants are using or anticipate using in future leadership roles, self-reflection was mentioned. Comments made relative to participant reactions to the self-reflective aspects of the program include: • “The self reflection aspect of the program was valuable because while I have reflected in my personal life, I've never really done that in my professional life. I 100 learned more about myself in one year in the GROW program than I have in all the other 33 years of being me” (I-1). • “It was also important to learn more about myself as a leader. I felt I already knew about myself, but the program helped me see how I would relate as far as part of an administrative team” (I-2). • “Most valuable to me was the opportunity for self-reflection. I valued the learning, but self-reflection had the most impact on me” (I-4). • “After the first few session, I thought to myself that I'm in over my head. In time it taught me to look at myself, how I teach, and how I relate to my students” (I-6). Theme–collaboration. Collaboration was another emergent theme that was reflected in responses during the analysis of coded interview transcriptions. Review of interview data supported collaboration as a valuable aspect of the GROW program. Sixty percent (n=6) of program participants reported the value of collaboration and the ongoing benefits of the relationships formed during the GROW program. Participant responses relative to collaboration included: • “The relationships I ended up building with my peers were very helpful because they were going through the same decision making processes that I was” (I-1). • “Working in a cohort added to the experience. We got to work with those across the district and learn about other schools” (I-2). • “It was fun and a great opportunity to collaborate with others. It complemented what I learned in facilitative leadership” (I-5). • “It was valuable to work in a cohort group” (I-8). 101 • “I most value the relationships I formed while in the program. I still rely on them today. I seek guidance and support from those with whom I worked in the program” (I-9). Theme–interest in leadership. One final theme that emerged from analyzing the coded responses from interview transcripts was the impact of the program on the interest level of participants in leadership. Fifty percent (n=5) of those interviewed discussed how the GROW program contributed to their interest in leadership and reported that the overall experience in the GROW program had a positive impact on their interest in advancing to a leadership role. Participants stated: • “The GROW program impacted my decision to apply for an assistant principal position. Once I started the GROW program I learned more about my abilities as a leader. It made me more serious about applying for leadership positions” (I-1). • “The program experience was beneficial. Because of GROW, I decided to apply for the MSA at UNC-W” (I-3). • “I applied for the MSA because of the GROW program. The GROW program provided a strong foundation for leadership. It provided a ‘booster shot’ for what I am learning in the MSA program” (I-8). • “The GROW program allowed me to explore the leadership qualities that others saw in me. The program tapped into my desire to lead. It caused me to step out of my comfort zone” (I-9). Another question to which interview participants responded regarding their reaction to the GROW program was designed to guide program improvement initiatives in Craven County Schools. Interview participants were asked their reactions relative to the program aspects that 102 were least valuable. Fifty percent (n=5) of those participants interviewed reported all aspects of the program as valuable. Comments included: • “There was nothing we did that was not valuable. I feel like we spent time on things that really affected me as a person” (I-1). • “Everything was valuable” (I-3). • “Everything had value. Everything was done with a purpose” (I-6). • “Honestly, that was nothing in the program that wasn’t valuable” (I-8). • “Everything had a purpose; even the jokes and dancing” (I-10). The remaining 50% (n=5) of participants interviewed provided program improvement suggestions for implementation in future GROW cohorts. Those participants stated: • “The least valuable aspect of the program was impact analysis because I already understood that every decision we make has an impact. I learned from it though” (I- 2). • “Some of the things we talked about I already knew, but I got something new out of everything we did” (I-4). • “There was a change that occurred within me, but I would have really liked feedback on the final presentation” (I-7). • “Continue to add more realistic situations. Coming back to some of the surveys would be helpful” (I-5). • “The surveys were the least valuable because there was not enough time to dig deeper into the surveys. Even if we had a website to go to in order to learn more about the survey results” (I-9). 103 The final interview question designed to address Kirkpatrick’s (2006) level 1 evaluation model was, “If you had to do it all over again, would you participate in the GROW Program? Why or why not?” The first part of the program provided quantitative data regarding participant satisfaction relative to their participation in the GROW program. Analysis of data revealed that 100% (n=10) of participants reported satisfaction with the GROW program by stating they would again participate in the GROW program, having knowledge of the curriculum and other program components. Satisfaction data is presented in Figure 10. In order to obtain detailed data necessary to determine why the participants were satisfied in the program, each participant interviewed was asked to elaborate on their closed ended responses. Their data provided the researcher with more specific information about of the factors and elements of the program that contributed to the high level of satisfaction with the GROW program. Participant responses included: • “It was a good experience. When I walked in, I thought I could sit back like in my college classes, but that didn't happen. Because of that, I learned so much about myself. The program allowed me to be myself. That was one of the most important parts of the program” (I-2). • “I believe the GROW program helped me move into a leadership position much faster. It's not just a ‘how to class’ it is a program about building you” (I-4). • “The experience was valuable. It pushed me to take the next step. GROW inspires you to do something bigger and better. The program just doesn't end. There's follow up and that keeps pushing me to the next level” (I-8). 104 Number of Participants Satisfied with GROW Program 0 2 4 6 8 10 YES NOParticipant Responses N um be r o f P ar tic ip an ts Figure 10. Bar graph showing participants’ satisfaction with GROW program. 105 • “It was a journey, not only professionally, but personally. Leadership is very personal. It is so important to understand that ‘best fit’ for you” (I-9). • “It was amazing to collaborate with others and watch them grow” (I-10). Reaction Summary (Emergent Themes) Kirkpatrick (2006), states that a favorable reaction to any training program is important since participants will share their reactions with others, impacting future program initiatives. The purpose of the first level of evaluation is to “determine how effective the program is and learn how it can be improved” (Kirkpatrick, 2006, p. 27). level 1 data was derived from two data sources; monthly assessments and participant interviews. Analysis of quantitative and qualitative data suggests an overall positive reaction of participants to the GROW program. In addition, a number of themes emerged during analysis of the reaction data. Emergent themes relative to reactions of the participants to the GROW program are self-reflection, collaboration, leadership, videos and tapping. Level 2 Evaluation: Learning Research question. What did participants in the GROW program learn relative to leadership principles? According to Kirkpatrick (2006), learning outcomes should also be assessed in a program evaluation. Level 2 of Kirkpatrick’s model is designed to measure the learning that occurred due to the training program. Assessment at this level moves beyond participant satisfaction and evaluates the extent that learning took place as a result of the GROW program. The program evaluator analyzed data from two sources to determine the level of learning that took place in regard to the eleven leadership principles addressed in the GROW program. Data sources for level 2 evaluation included a GROW evaluation and individual participant interviews. 106 GROW Evaluation. The program evaluator in this study administered a retrospective post-then-pre evaluation to measure individual growth based on participant knowledge level relative to the leadership principles addressed in the program. The evaluation asked participants to rate pre and post knowledge of the core leadership principles using a 7-point Likert-type scale. Participants were asked to rate their level of knowledge of each leadership principle based on their knowledge prior to and at the conclusion of the GROW program. The scale ranged from a “1”, signifying little knowledge, to a “7”, signifying a great deal of knowledge of the leadership principle. The analysis of pre- and post-surveys provided data on the level of knowledge participants reported in relation to the following eleven leadership principles: • Know Thyself • Core Values • Courage • Know They Territory • Building a Culture • Goal Setting • Communication • Decision Making • Impact Analysis • Conflict Management • Accountability The results of the retrospective post-then-pre evaluation of level of knowledge are presented in Figure 11. The mean of the participant pre-assessment levels are represented in blue bars, while the mean of the post-assessment levels are represented in the red bars. As evidenced 107 Group Average of Post-then-Pre Assessment 0 1 2 3 4 5 6 7 Kn ow Th yse lf Co re Va lue s Co ura ge Kn ow Th y T err ito ry Bu ild A Cu ltu re Go al Se ttin g Co mm un ica tio n De cis ion M aki ng Im pa ct A nal ysi s Co nfl ict M an age me nt Ac cou nta bil ity Leadership Principles Le ve ls o f U nd er st an di ng Pre Assessment Post Assessment Figure 11. Bar graph showing pre- and post-evaluation results. 108 in the graph, the mean of the post-assessment results illustrate growth in knowledge of each of the learning principles. The pre-assessment mean scores ranged from 2.5 (Impact Analysis) to 4.0 (Accountability). The post-assessment mean scores ranged from 5.0 (Impact Analysis) to 6.2 (Communication). The program evaluator further analyzed participant responses in regard to their growth in level of knowledge and understanding by leadership principle. Figure 12 illustrates the range of average growth by leadership principle. The analysis is arranged from the topic about which the most reported increase in knowledge occurred, to the topic with the lowest level of reported increase in knowledge. The leadership principles about which the participants reported the highest mean gain of 2.6 are conflict management and know thy territory. Other leadership principles showing a reported high level of increased knowledge were impact analysis (average 2.5 increase) and courage (average 2.4 increase). Participants reported the lowest mean gain in increased knowledge and understanding of 1.9 in goal setting. Overall, the average growth reported by participants in all leadership principles was 2.2. Participant survey data supports a high level of learning relative to participant knowledge and understanding of the leadership principles as a result of participating in the GROW program. Participant interviews. The level 2 evaluation was conducted in order to determine what participants reported to have learned as a result of participating in the GROW program. The interview protocol was structured to include four questions designed to probe participants regarding perceived learning obtained during the ten months of the GROW program. Participants were asked to respond to the following prompts aligned with the learning level in Kirkpatrick’s evaluation model (2006): 109 Average Growth and Percent Improvement by Leadership Principle 1.9 2.0 2.1 2.2 2.2 2.3 2.3 2.4 2.5 2.6 2.6 0 0.5 1 1.5 2 2.5 3 Goal Setting Know Thyself Accountability Communication Core Values Decision Making Build A Culture Courage Impact Analysis Know Thy Territory Conflict Management Le ad er sh ip P rin ci pl es Average Growth Figure 12. Bar graph showing growth by leadership principle. 110 • What leadership principles do you feel you most developed while participating in the GROW program? • Do you believe the GROW program will help develop a stronger pool of applicants for school leadership positions? Why or why not? • As part of the program, you learned about your leadership style. Share with me what you learned about yourself as a leader and how this has impacted how you lead others. • Describe the learning that took place for you as a result of participating in the GROW program. The first level 2 question was designed to determine which leadership principals were most developed as a result of the GROW program. Analysis of the data revealed two common themes in the responses of participants: courage and decision making. Theme–courage. Thirty percent (n=3) of those participants interviewed stated courage as the leadership principle that was most developed in the GROW program. When asked how the GROW program helped her develop courage, one participant stated, “I developed courage; the courage to apply for a leadership position, the courage to be a leader, and the courage to have crucial conversations. I learned I can get through just about anything. I just need to believe in myself” (I-1). Theme–decision making. Another leadership principle that participants reported developing most is the ability to make decisions. Thirty percent (n=3) of GROW participants reported this area as most developed and addressed the importance of the decision making process. One participant interviewed stated, I learned that all decisions must be based on data. 111 Everything has to be backed up with data” (I-5). Another stated, “I now understand the impact of the decisions you make on people and on the school as a whole” (I-9). Another interview question in level 2 was developed to determine if the GROW program helped develop a more qualified pool of candidates for school leadership positions based on learning outcomes of the program. Of the ten participants interviewed, 100% reported to believe that the GROW program will provide more qualified candidates for leadership positions in Craven County Schools. The data is presented in Figure 13. In order to determine what participants perceived as the reason the GROW program will help to develop a more qualified pool of applicants, the researcher asked participants to explain how the program will accomplish this outcome. Program participants gave the following responses: • “Even in my masters, I wasn't sure I was meant for administration. The GROW program teaches you to be real. It was very helpful to me” (I-2). • “We will have stronger leaders within the district. We have built a cohort of support and that helps you become more comfortable with becoming a leader” (I-3). • “It helps with content knowledge, growth inside of participant, and the group relies on each other for support” (I-4). • “By identifying the leadership potential in our schools, now you have a good place to start affecting change” (I-7). • “You learn about school level leadership, but also a systems perspective. GROW provides you with a tool kit to take to the next step” (I-8). • “Participants leave the program with more knowledge about leadership” (I-10). 112 Number of Participants Reporting Positive Impact of GROW Program on Candidate Quality 0 2 4 6 8 10 YES NOParticipant Responses N um be r o f P ar tic ip an ts Figure 13. Bar graph showing impact of GROW program on increasing the pool of qualified candidate for leadership positions. 113 The final two interview questions were developed to gain a more comprehensive understanding of the learning reported by participants. Respondents were asked to share information about personal learning and leadership style. Participants reported a variety of learning experiences and discoveries related to leadership. Some of the responses shared by interview participants are as follows: • “I learned that I lead by teaching and that I grow others. I am stronger and more confident in my observations and conversations with teachers. I learned that leadership is about messing with people's lives” (I-4). • “I learned that as a leader I'm messing with people's lives and messing up people's lives. I also learned how to lead strategically” (I-6). • “What I really liked was the real life scenarios and the case studies used to provide us with a pragmatic learning approach. I learned more by failing at those activities than any other activity” (I-7). • “I learned the importance of transparency in leadership. As leaders, we need to explain the reasoning behind our decisions. I think the secret is learning to blend perspectives” (I-9). • “I learned that as a go getter, I tend to dominate the conversation. I need to sit back and listen. I now see I can learn when I listen” (I-10). Learning Summary (Emergent Themes) Kirkpatrick (2006) stresses the importance of measuring learning after the training program. “Learning can be defined as the extent to which participants change attitudes, improve knowledge, and/or increase skills as a result of attending the program” (Kirkpatrick, 2006, p. 27). Level 2 data addressed each of these areas through the use of a retrospective post-then-pre 114 evaluation at the end of the program and participant interviews conducted eight months after the conclusion of the program. Analysis of data regarding what participants reported as learning outcomes of the GROW program suggested the program was beneficial to individual participants and the district. Participants reported learning and the development of a new awareness of leadership competencies due to program participation. The data revealed several common themes. Emergent themes relative to participant learning in the GROW program are impact analysis, courage, decision making, and collaboration. Level 3 Evaluation: Behavior Research question. How has the behavior and attitudes of the participants in the GROW program relative to school leadership changed as a result of participating in the program? Kirkpatrick (2006) defines behavior evaluation as “the extent to which change in behavior has occurred because the participants attend the training program” (p. 22). The purpose of conducting a level 3 evaluation is to determine not only what behavior has changed, but how participants are using the new knowledge to impact their work. The data collected to conclude the impact of the GROW program on participant behavior was derived from two primary sources, a summative questionnaire completed at the conclusion of the training and individual participant interviews. Interviews were conducted eight months after program completion I order to give participants ample time to implement changes in their work settings based on program participation. Summative questionnaire. A summary questionnaire was administered at the conclusion of the GROW program. All responses were anonymous. The questionnaire included questions relative to behavior changes of the participants and was developed to determine: 115 • If program participation increased participant interest in advancing to leadership roles. • How the program shaped the beliefs of the participants. Several questions on the questionnaire were designed to determine the behavior changes of the participants of the GROW program. Quantitative data was derived from two questions on the survey. One question was designed to determine if the program increased the interest of participants in advancing to a leadership role by having each survey participant select from a Yes/No option. Twenty-one participants responded to the survey question, “Did program participation increase your interest in advancing to a leadership position?” Based on analysis of the responses, 95% (n=20) reported an increase level of interest in school leadership due to program participation, while 5% (n=1) report no increase in interest. Data collected is presented in Figure 14. In order to gain a more comprehensive understanding of the impact of the program on the interest level of participants in advancing to a leadership position and to address the “so what” Michael Patton (2008) proposed as critical in program evaluations, comments of participants were analyzed to look for patterns in their responses. While there were general comments that supported a relationship between the program and an increased level of interest in leadership such as, “The program increased my interest in advancing to a leadership role because it has given me more insight into what it takes to be a leader, as well as strategies to use while I'm a leadership position” (Q-6) and “The program caused me to really examine myself and how I might fit into leadership roles” (Q-14), two common themes were found in participant comments; realized potential and self-confidence. 116 Increased Level of Interest in Advancing to a Leadership Role 19 1 0 2 4 6 8 10 12 14 16 18 20 YES NO Participant Responses N um be r o f P ar tic ip an ts Figure 14. Bar graph showing change in interest in leadership. 117 Theme–realized potential. Thirty-eight percent (n=8) of participants made comments relative to having an improved understanding of and belief in their leadership potential. Participants stated: • “The program helped me to realize my potential as a future administrator” (Q-4). • “I became more aware of the leadership potential I possess” (Q-5). • “The program highlighted strengths that I did not realize I possess. It took something that seemed foreign (leadership) and made it seem attainable” (Q-7). • “GROW allowed me to dig deeper into my true talents and abilities to lead” (Q-13). • “In learning about all that is involved to be a successful leader, I found that I am far better equipped than I thought at the beginning of the program” (Q-19). Theme – self-confidence. Twenty-nine percent (n=6) of respondents report more self- confidence due to participating in the GROW program. Statements relative to the impact of the program on the confidence level of participants include: • “I have an increased interest in leadership because I feel more prepared” (Q-8) • “The program helped me learn about my leadership style and I am more confident” (Q-9). • “When I started the GROW program, I wasn't sure administration was the direction I wanted to go. Now, I have more confidence in myself and know that administration would be a good fit for me” (Q-10). • “I found my voice” (Q-12). • “I have learned more about how to be an effective leader and gained confidence in my ability to lead” (Q-16). 118 Using a three level Likert-type scale, a second question on the summative questionnaire was designed to determine the intent of the participants to apply for a school leadership positions within the next five years. The survey provided the options of More Likely, Less Likely or No Change. Of the 21 respondents, 71% (n=15) reported more likelihood of applying for a school leadership position within the next five year. No respondents reported being less interested in pursuing a leadership positions within five years, while 29% (n=6) reported no change in their intent to apply for leadership positions within the next five years. Data is presented in Figure 15. Participant interviews. Participant interviews were conducted eight months after the completion of the GROW program in order to give participants an opportunity to implement learning that changed the leadership behavior of participants. The interview protocol included six questions designed to assess behavior changes in the participants. The behavior questions asked of the 10 participants purposively selected to participate in the individual interview were designed to determine: • How participants have applied learning from the GROW program to their daily work. • The aspect(s) of the program that most influenced the beliefs of the participants. • The topics in the program that most changed the behavior of the participants. • If the participants’ interest in leadership changed. • If participants are more or less interested in becoming a school leader since participating in the GROW program. • The future plans of the participants relative to school leadership. Three questions to which participants responded during the interview were related to behavior changes on the job. The researcher was interested in determining the extent the participants applied the learning reported as a result of the GROW program. Thematic analysis 119 Level of Interest in Leadership 0 3 6 9 12 15 18 21 Level of Interest N um be r o f P ar tic ip an ts Number of Responses 15 0 6 More Interested Less Interested No Change Figure 15. Bar graph showing level of interest in leadership after program participation. 120 Three questions to which participants responded during the interview were related to behavior changes on the job. The researcher was interested in determining the extent the participants applied the learning reported as a result of the GROW program. Thematic analysis of the coded responses of participants generated emergent themes aligned with behavior changes in the daily work of GROW participants. The most common responses related how participants’ behavior has changes were relative to decision making, building relationships, and knowing thy territory. Theme–decision making. Decision making was the most commonly addressed area of behavior change in the daily work of the participants. Decision making was referenced nine times by 60% (n=6) of GROW participants interviewed. Comments from interview participants addressing changes in the way decision are made based on learning in the GROW program include: • “I'll never forget that Dr. West said, ‘You're messing with people's lives and you're messing up people's lives’. That makes you think twice before you jump to a decision” (I-1). • “I use what I learned to serve on school cabinet and in the decision making process” (I-5). • “I use the decision making process on a daily basis” (I-6). • “I am finding I have a growing appetite for school decision making. I’m not so much a Monday morning quarterback anymore, but I’m taking a professional interest in how we go about the business of public education” (I-7). • “The criterion is just as important as the decision you make. You must have a rationale behind your decisions” (I-10). 121 Theme–building relationships. Another theme that emerged from the coded interview data was the changes in how participants build relationships with their co-workers. The changes in behavior regarding how participants focus on building relationship was addressed by 60% of participants (n=6) and mentioned seven times in total responses. Comments supporting the change in how participants build relationship include: • “My communication is more open because I've learned how to better work with people” (I-2). • “Building the ‘sprit de corps’ within my classroom and program by using jokes, stories and real conversations has helped me reach and teach my students more effectively” (I-7). • “I am a better listener to the people around me. When you listen to people, you know the pulse of that school” (I-9). • “I am empowering others by constantly encouraging greatness. As leaders, we are tapping into people every day” (I-9). Theme–know thy territory. Knowing thy territory was another emergent theme when participants discussed their behavior changes since participating in the GROW program. Coding responses enabled the researcher to collect data regarding knowing they territory as a common theme. Knowing thy territory was addressed by 40% of those interviewed (N=4) and mentioned seven times during the focused interview question responses. Interview participants made the following statements: • “I'm learning new territories. It made me try to do a better job as a leader. Today is a different world than it was 10 years ago. You have to know the territory of those around you” (I-2). 122 • Know thy territory. It is so important to know who I work with” (I-3). • “I've focused on learning my territory. This has helped me transition into my new role as an assistant principal” (I-6). The second set of interview questions used to examine if behavior changes in the participants were related to their plans regarding becoming a school leader. Of the 10 participants interviewed, 100% (n=10) reported an increased interest in becoming a school leader. Participants also gave examples of how their behavior had changed or will change due to the increased interest level in school leadership. Participants stated: • “I became more passionate and excited about becoming a leader. At first it was about other people telling me that I could lead, but now I believe it” (I-1). • “I'm more interested in leadership. I've applied for two leadership positions within the district” (I-2). • “As far as my interest in leadership, a maybe has turned to a yes” (I-3). • “The GROW program encouraged me to do more as a leader. I have applied for the MSA program at ECU and hope to hear back about my acceptance in May” (I-5). • “My interest in leadership was transformed into a passion because of the GROW program. I am now an assistant principal at a high school” (I-6). • “I would like to explore different options in school leadership to find out what I like and don’t like. I want to build a culture of togetherness, a culture of innovation, and a school culture where all students believe they can be successful” (I-7). • “I am more interested in leadership and have taken an assistant principal position. I gained not only an interest in leadership, but an appreciation for it” (I-8). 123 • “I am more interested in leadership, but I need to take baby steps and lead in my school before considering a positions like assistant principal” (I-10). Behavior Summary (Emergent Themes) According to Kirkpatrick (2006), level 3 evaluation is often difficult because decisions regarding when and how to evaluate are complicated. For this reason, the evaluator in this study used two evaluation measures that were administered eight months apart. Using two evaluation measures given months apart gave short-term and long-term data regarding behavior changes in GROW program participants. Kirkpatrick (2006) also stated, “Tabulation of the responses can provide a good indication of changes in behavior” (p. 62). The coding and tabulation of response frequency in evaluation level 3 suggested positive changes in participant behavior with emergent themes support by thematic data analysis. Emergent themes impacting behavior changes reported by participants included an increased interest in leadership, improved self-confidence, realization of leadership potential, better skills in decision making, better understanding of building relationships, and knowing the territory in which they work. Level 4 Evaluation: Results Research question. Did the GROW program meet the goal of designing and implementing a program effective in increasing the pool of qualified and competent leaders in Craven County Schools? In Kirkpatrick’s (2006) evaluation model, the intent of level 4 is to determine the results of the training program. According to Kirkpatrick (2006), level 4 evaluation “leans heavily on the word ‘evidence’, as it is rare that you can limit factors enough to create true proof” (p. 113). In this study, the program evaluator is attempting to build a profile relative to program results using multiple evaluation methods. The methods were designed to measure two potential 124 outcomes of the GROW program implemented in Craven County Schools. First, data was analyzed to determine program effectiveness based on participant perceptions and reports of implementation of program principles in their daily work. Second, data was analyzed to determine program impact on the organization. Kirkpatrick (2006) describes this impact as the return on investment made by the district. Level 4 attempts to answer the question, “Did the program do what it was intended to do?” In this study, the data was analyzed to determine if the GROW program produced an increase in the number of qualified and interested candidates for leadership in Craven County Schools. Finally, data was analyzed to identify potential programmatic improvements. Data sources analyzed as part of level 4 evaluation measures included a summative questionnaire completed at the end of the training and individual participant interviews conducted eight months after the conclusion of the GROW program. Summative questionnaire. A summary questionnaire was administered at the conclusion of the GROW program. All responses were anonymous. The questionnaire included three questions relative to the results of the GROW program on the current and future work or the participants and benefits of the program to the school district. The questions were designed to determine: • If participants feel the GROW program helped develop a larger pool of qualified applicants for school leadership positions. • The elements participants identify as most effective of the GROW program. • If participants feel more prepared to become leaders in their schools as a result of participating in the GROW program. 125 One question was designed to provide quantitative data regarding the participants’ opinions as to whether the GROW program helped develop more qualified candidates for leadership positions in Craven County Schools. Each survey respondent was asked to reply to the statement, “Do you feel the GROW program will help develop a stronger pool of applicants for school leadership positions?” The participants were asked to respond in the affirmative or negative. Twenty-one participants responded to the survey question. Based on analysis of the responses, 100% (n=21) reported to believe the GROW program will contribute to a stronger pool of applicants for school leadership positions. Data collected is presented in Figure 16. In order to gain a more comprehensive understanding of the relationship between the GROW program and a more qualified pool of applicants for school leadership positions, participants were asked to elaborate on each response by answering the question “Why or Why not?” Nineteen of the 21 survey participants elaborated on the Yes/No response selected. Using thematic analysis, comments were coded in order to identify common patterns and themes in participant comments. Several emerging themes were evident in the coded comments. Theme–self-reflection (question 1). Participant comments referencing self-reflection as a reported reason the GROW program will help build a more qualifies candidate pool for leadership positions occurred in 47% of participant comments (n=9). This was the most common occurring theme in the data set. Participants’ comments related to self-reflection included: • “This program asks you to delve deep into yourself and look at what you can do and what you need to do in order to be effective” (Q-3). • “I believe the program really makes you look at your leadership style and understand what you need to work on to be an effective leader” (Q-9). 126 Number of Participants that Feel GROW Program Contributed to a Stronger Candidate Pool for Leadership Positions 0 3 6 9 12 15 18 21 N um be r o f P ar tic ip an ts Number of Responses 21 0 Yes No Figure 16. Bar graph showing participant opinions regarding contribution of the GROW program to a stronger pool of applicants for school leadership positions. 127 • “The best part of the program was how we had to reflect on ourselves and every individual need self-reflection in order to grow” (Q-10). • “The program opens your eyes to many of the challenges of school leadership and encourages leaders to engage in self-reflection and evaluation” (Q-16). • “Reflection is such a powerful tool. This program encouraged reflective thinking and planning with regards to all aspects of leadership. It also afforded participants the time to set goals based on that reflection. I know that I am personally surprised to learn that leadership skills already existed in me” (Q-19). Theme–leadership principles (question 1). Thematic analysis of participant responses also revealed the common theme of leadership principles. References to the increase of knowledge of leadership principles as having a positive impact on developing a more qualified pool of administrative candidates occurred in 32% (n=6) of participant comments. Comments referencing the leadership principles included: • “The leadership principles we learned about are important for leaders to possess and be aware of. This program does a good job of making participants more aware of the principles. I believe it also helps participants enhance these skills” (Q-5). • “The program gives the participants the "essentials" for being a school leader. (i.e. best practices, strategies, core values, principles, etc.)” (Q-6). • “GROW provided administrative candidates with a stronger foundation of leadership qualities and facets that will truly help them decide if this is the path they want to take” (Q-13). Theme – collaboration (question 1). Occurring in 26% (n=5) of the participant comments was the theme of collaboration. Using words such as “support group”, “network”, 128 and “connections”, participants reported collaboration as an element of the GROW program that contributed to a stronger pool of applicants for school leadership positions. The theme of collaboration was addressed by participants in the following comments: • “Dr. West brings confidence and allows for participants to have a support group from the start” (Q-4). • “The GROW program creates a network of people who have an interest in leadership and it nurtures a process that allows them to see themselves in that role” (Q-7). • “The program also has allowed us to meet other future leaders and make connections that will help us” (Q-14). Further thematic analysis identified additional common themes including gaining confidence, high quality of presenters, and inspiration as having a positive impact of developing future leadership applicants; however, they were not as well supported in quantity of responses. The next question designed to address the results of the GROW program on the summative questionnaire was, “What does the program do most effectively?” To further prompt participants, respondents were asked to address the parts of the program that have most transformed participants as leaders. While several comments, such as “The GROW program takes something set apart and breaks it down to more attainable, understandable parts with which I can identify” (Q-8), “The GROW program identifies what it means to be a leader” (Q-19), and “The GROW program awakened my passion for leadership” (Q-13), analysis of all coded comments yielded common themes in participants responses. Theme – self-reflection (question 2). The most common referenced theme was self- reflection. Of the 21 comments, 43% (n=9) indicated self-reflection as being an effective and a transformative element of the GROW program. Participant comments included: 129 • “The program has made me an even more reflective practitioner. The camaraderie, discussions, and reflective practices helped transform me as a leader” (Q-7). • “The most effective aspect of the GROW program is the self-reflection. While I reflect on my teaching, I don't take enough time to reflect on myself. This program is an excellent gut-check to help each person decide if they are up for the challenge of leadership” (Q-11). • “The program encourages us to examine ourselves and our leadership styles. We were given lots of opportunities to find out more about our strengths and weaknesses and to look for ways to build on our strengths and improve our weaknesses” (Q-17). • “The reflection piece was most effective to me. Very rarely are we afforded the time to reflect deeply before, during, and after learning. We were given that opportunity at EVERY session. That time to reflect and evolve was definitely transformative” (Q- 20). • “The program allows you to grow as a person and to better know yourself” (Q-22). Theme–leadership principles (question 2). Another theme that participants stated as a transformative component of the program was gaining knowledge of the leadership principles. Thirty-three percent of respondents (n=7) noted the leadership principals as having a positive impact on the results of the program and most transformative for program participants. Participants made the following comments in relation to the positive results aligned to the leadership principles: • “The leadership principles (courage, impact analysis, decision making process) helped me prepare for an AP position” (Q-3). 130 • “The program does a great job of making participants aware of the principles of leadership” (Q-6). • “The leadership principles definitely transformed me, helping me evaluate areas I had not considered” (Q-11). • “The pace of the program and discussion of principles in depth have allowed me to really grasp the principles of leadership. The analogies, video, and group work created opportunities to connect to concepts” (Q-14). Theme–collaboration (question 2). A final theme that emerged as participants disclosed what they reported to be the most effective and transformative aspect of the program was collaboration. Of the 21 participants that responded to this survey question, 29% (n=6) stated that collaboration opportunities that were intentionally designed and fostered in the GROW program were transformative. The theme of collaboration was evident in the following responses: • “The program did a great job allowing dialogue and creativity and developed a sense of community among participants” (Q-4). • “I was able to listen to my colleagues' ideas and beliefs and in turn, it helped me solidify or slightly alter my own beliefs” (Q-7). • “The program allowed us to collaborate with our peers and share ideas on leadership in a comfortable setting” (Q-9). • “Most transformative for me was the interactions with the people in the group” (Q-21). One final question on the summative questionnaire developed to address the results of the program was intended to determine how well the program prepared participants to lead in a 131 school. The respondents were asked to give an “overall” view of their personal preparation to lead based on participation in the GROW program. Respondents generally made positive comments as to their preparation for leadership based on participation in the GROW program. Among the 21 participant comments, responses ranged from prepared (38%) to very well prepared (57%). Some of the comments regarding feeling more prepared for leadership were: • “I am prepared as one can be outside of the actual experience in a leadership position” (Q-5). • “I feel very prepared for leadership because I know my strengths and areas I need to improve in to be effective” (Q-9). • “I have already held some leadership roles, but now I am ready to take the next step toward administration. I fell much more prepared now than when I started the GROW program” (Q-11). • “I am more prepared for school leadership than when I started the program” (Q-12). • “I feel like I'm now ready to take the next step in this journey. I'm fully prepared and will be able to grasp leadership better” (Q-14). • “The program prepared me as much as anything could. This program laid the foundation for leadership” (Q-15). • “I feel very prepared to become a leader. Actually, I found that I am already a strong leader; I just needed to find the courage and confidence within me to take the next step. Now that I've applied for a leadership position, I can't wait to get a position. This is NOT how I felt at the beginning of the program” (Q-20). Three participants had more unique, yet valuable responses related to the impact of the GROW program on leadership preparation. The first two responses suggested that participants 132 feel less prepared. While this may appear to be a negative reaction, further review of the elaboration of the response with additional comments suggests that the participants feel less prepared due to gaining a deeper understanding of exactly what school leadership entails. One participant stated, “I actually feel less prepared, if that's possible. The program pointed out my shortcomings. I am quite confident that I am NOT prepared to be an administrator. However, I feel more prepared for other leadership positions” (Q-6). A second participant stated, “The program has shown me that I have a ways to go and that my journey is far from over, but I know that I can do it” (Q-4). One final participant suggested the GROW program was not about preparation by stating, “Becoming a leader is a process. The GROW program is more about "shaping" leaders than preparing them” (Q-19). Participant interviews. In order to gain a more comprehensive understanding of the results of the GROW program on participants, interviews were conducted eight months after the completion of the program. Since this program evaluation is based primarily on participant perceptions and reporting, it is difficult to provide tangible results based solely on numbers. For that reason, nine interview questions were designed to address the results of the program relative to the program outcomes impacting individual participants and the school district. This level of evaluation focused on determining if the program objectives were met based on participant reports. The interview data was also used to gather participant suggestions for program improvement. Three questions were designed to determine results of the program on the participants. Program participants were asked to share ways the program has produced results in the work of the individual participants either through improved performance or in their transition to a leadership position since participating in the GROW program. Ten program completers were 133 interviewed and 100% (n=10) of the interview participants reported positive results relative to their professional work as individuals. Comments reported by interviewed participants that support the results of the GROW program included: • “I'm a new assistant principal. I applied because I'm more aware of my abilities to tap into the leadership of others. I'm more confident” (I-1). • “I am communicating more effectively. I ask questions. I've taken on more leadership roles. I'm not afraid of leadership. I have found the courage to lead” (I-2). • “Because of the GROW program, I am more likely to seek a position as an assistant principal” (I-3). • “I am more willing to apply for a leadership positions. I have already applied for the MSA program” (I-5). • “I’ve transitioned to an assistant principal position and I am using my strengths to effectively support teaching and learning. I not only use my strengths, but the strengths of others. I am more strategic” (I-8). • “I am now an assistant principal of a middle school” (I-9). • “I have implemented what I learned at work and at home. I have had conversations I would not have had before. I am facilitating meetings and getting better at the leadership roles I already had” (I-10). Two interview questions sought information from participants regarding the return on investment for the school district. Participants were asked about the benefits to the school district and the success of the program in providing more qualified candidates for leadership positions in Craven County Schools. Of the participants interviewed, 100% (n=10) reported that they were more likely to apply for leadership positions within the district since participating in the GROW 134 program. Participants also reported additional benefits to the school district. Benefits to the district suggested by participants included: • “The district should continue the GROW program. You build relationships in the MSA program, but in the GROW program they're people right here in the district. Together we have been able to tap into potential that we didn't even know was there” (I-1). • “The district will have more buy in now that we know about leadership principles and the decision making process” (I-2). • “The cohort model helps the district by providing a group of people to go to. It also develops a pool of leaders” (I-3). • “The GROW program will produce stronger leaders and stronger leaders will produce stronger schools” (I-4). • “One benefit of the program is the proactive approach the district has taken to build leaders” (I-5). • “I think GROW will create leaders people want to work with. For the district, GROW will produce more effective leaders, improve morale, and impact instruction” (I-8). • “The program opened doors that have never been there before. Through these doors, more district leaders will emerge” (I-10). Participants were asked to respond to the question, “What can be done to enhance the quality of the program for future participants?” The purpose of this interview question was to get participant input regarding program improvement suggestions in order to continuously improve district leadership development initiatives. Suggestions from participants interviewed 135 included four comments regarding the opportunity to shadow or connect with leaders in order to gain a better understanding of what school leadership is about on a daily basis. Those comments included: • “Incorporate shadowing opportunities outside of our buildings to see what leadership is really about” (I-1). • “It would be helpful to shadow leaders” (I-6). • “GROW participants that have moved into leadership positions could follow up with the cohort to talk about our leadership experiences. We can share our experiences because there are things you simply can't know until you’re in the position” (I-8). • “Include opportunities to shadow a leader and actually spend the day walking in the shoes of a principal. There's nothing like actually being there to see what it’s like firsthand” (I-9). Another program suggestion that emerged from participant interviews addressed the use of authentic examples in the GROW program. Based on analysis of responses, 30% (n=3) of participants value authentic learning experiences that apply to their daily work. Comments supporting authentic scenarios and activities included: • “Continue to make the program real world. I'm in a MSA program and it’s about papers and processes, but not real preparations for the job. The GROW program is about the reality of the job” (I-4). • “Continue to use real life scenarios” (I-5). • “What I really liked was the real life scenarios and the case studies” (I-7). 136 Review of District Employment Data In order to further determine the impact of the implementation of the GROW program in Craven County Schools, district employment data was analyzed. Data was reviewed to determine the number of GROW participants that have transitioned into school leadership roles. A review of the reassignment data indicated that 25% (N=6) of the 24 completers of the GROW program have transitioned from a teaching position to an assistant principal positions and two other completers have transitioned to lead teacher roles (R. Kelley, personal communication, January16, 2015). This data supports the effectiveness of the GROW program in increasing the number of qualified candidates for school leadership positions in Craven County Schools. Results Summary (Emergent Themes) According to Kirkpatrick’s (2006) level 4 evaluation, it is often challenging when results are not tangible, so a program evaluator should seek evidence rather than proof. Using multiple evaluation methods to measure the results of the GROW program helped to build a chain of evidence relative to program outcomes. A summative questionnaire was completed at the conclusion of the program and participant interviews were conducted eight months later to study the ongoing benefits of the GROW program for the participants and the school district. The results of the data suggest positive results of the GROW program for both participants and the school district. These results can be used to justify the return on investment for the school district and to seek grant funding for future program initiatives. An analysis of common responses revealed from coded data relative to results indicated emergent themes in courage, collaboration, and interest in leadership. 137 Impact of GROW Program (Emergent Themes) Analysis of all data collected across each of Kirkpatrick’s (2006) four levels of evaluation provided evidence regarding program outcomes necessary to answer the primary research question of this study; How effective is a school system’s research-based Grow Your Own leadership development program in growing a pool of qualified candidates to place in future leadership positions? Results were provided based on an analysis of both the quantitative and qualitative data for the study. A determination as to the effectiveness of the implementation of the GROW program in Craven County Schools was evaluated on the degree to which extent participants reported a positive reaction to the program, learning that occurred, as well as behavioral changes reported by participants Each evaluation level contained data that has been analyzed and interpreted to determine the following quantitative results: (1) Participants had a favorable reaction to the GROW program, (2) Participant survey data supports a high level of learning relative to participant knowledge and understanding of the leadership principles as a result of participating in the GROW program, (3) Quantitative data suggests participant interest increased relative to advancing to a leadership position, and (4) Participants of the GROW program are more likely to apply for school leadership programs since participating in the program. The primary objectives of the GROW program was to provide more qualified candidates for leadership positions in Craven County Schools. Based on participant feedback, the program was highly successful in meeting that goal. Qualitative results were interpreted through the analysis of patterns, themes, and categories of evaluation data. Based on thematic analysis, the program produced positive results 138 with several overarching themes as noted in Table 7. Participants reported a positive impact of the program based on the value of program components that emerged as themes. The importance of collaboration was found to be significant in all levels of Kirkpatrick’s (2006) Evaluation Model. Leadership emerged in the levels of reaction, behavior, and results. Self-reflection and self-confidence was also a common theme and was noted in both the reaction and behavior level. Another area that participants reported as having a positive impact is the decision making process, which occurred in the behavior and learning levels. One final theme that emerged from analysis of data from the evaluation levels was courage. Courage was noted as important in the learning and reaction levels of Kirkpatrick’s model. Based on an analysis of quantitative and qualitative data, the GROW program was successful in providing more qualified candidates for leadership positions in Craven County Schools. Participant comments support the positive results of the program. One participant stated, “Because of the GROW program, I felt better prepared for real life leadership” (I-9). Another noted, “The program creates leaders that are empowered to lead. It is also creating role models for teachers and students” (I-6). The transformative power of the GROW program is illustrated in the comment of a GROW participant who wrote: I've had many different "types" of administrators in my 13 years. I've learned a lot about what to do and maybe more about what not to do. My fear to this point has been that if I took the next step, I would become too far removed from the classroom, losing sight of what's most important...our children and their holistic well being. After learning about the leadership principles and seeing it is possible to lead with a ‘teacher's heart’, my beliefs about being a school leader, in an administrative role, have changed. The program reminded me that if I have the courage to hold myself accountable and stay true 139 Table 7 Themes in Data by Kirkpatrick’s Level Kirkpatrick's Level Themes Level 1: Reactions 1. Self-reflection 2. Collaboration 3. Leadership 4. Videos 5. Tapping Level 2: Learning 1. Impact Analysis 2. Courage 3. Decision Making 4. Collaboration Level 3: Behavior 1. Interest in Leadership 2. Self-confidence 3. Leadership Potential 4. Decision-making 5. Building Relationships 6. Knowing Thy Territory Level 4: Results 1. Courage 2. Collaboration 3. Interest in Leadership 140 to myself, always upholding my core values, I have a lot to offer as a school or district leader (Q-19). Summary Chapter four presented an analysis of the data collected regarding the implementation of the GROW program in Craven County Schools. Data on the program outcomes was presented in tables and in narrative form. In addition, emerging themes were discussed across the four levels of Kirkpatrick’s Evaluation Model (2006). The program evaluator analyzed the data from four sources to answer the secondary research questions that contributed to addressing the primary research question in this study regarding the effectiveness of a school system’s research-based Grow Your Own leadership development program in growing a pool of qualified candidates for future leadership positions. The evaluator was able to determine that the GROW program had a positive impact on program participants and contributed to growing a pool of qualified candidates to fill future leadership positions. As a result of the GROW program, participants report an increased interest in leadership and 25% (N=6) of the 24 completers of the program have transitioned from a teaching position to an assistant principal positions. Two other participants of the program have transitioned to lead teacher roles. Further discussion regarding the findings, conclusions, recommendations for programmatic improvement, limitations, and recommendations for future research will be presented in Chapter Five. CHAPTER FIVE: SUMMARY, CONCLUSIONS AND RECOMMENDATIONS Introduction Chapter Five of this program evaluation study includes a summary of the research, discussion of research findings, and conclusions drawn from data presented in Chapter Four. Following the conclusions, recommendations for programmatic improvement, limitations, recommendations for further research and concluding remarks will be highlighted. Summary As indicated in Chapter One, the purpose of this study was to evaluate the leadership development program designed and implemented in Craven County Schools. This program evaluation was designed to determine if implementation of a leadership development program designed to assist future leaders to Gain skills, Realize potential, Optimize Impact on Others, and increase Work Satisfaction (GROW), had a positive impact on program participants. In addition, the program evaluation study was developed to determine if the implementation of the GROW program addressed the problem of practice relative to the lack of qualified and interested leadership candidates currently within the district. The evaluator was also interested in determining if the completers of the district-designed Grow Your Own leadership program had an increased level of interest in securing school leadership positions. In order to better understand the research related to the problem of practice outlined in this study, the program evaluator conducted a literature review. Since research relative to Grow Your Own school leadership programs in education is sporadic, literature was reviewed in both the fields of education and business. Review of the literature revealed data on the shrinking pool of candidates, thus the need for supporting the development of school district initiated Grow Your Own leadership development programs. Research regarding the components of the 142 program was also reviewed including program design and elements, developing a competency framework, building a candidate pool, conducting a needs assessment, recommended program components, and evaluation of program effectiveness. Through the review, Donald Kirkpatrick’s Model of Program Evaluation (2006) was selected as the evaluation model for this study. A review of the literature provided the foundational research used in this program evaluation study. This study used an action-oriented program evaluation research model and utilized a mixed methodology design to provide a chain of evidence necessary in determining the success of the implementation of a district Grow Your Own teacher leadership program in Craven County Schools. The evaluation component used in the study was based on Donald Kirkpatrick's (2006) Model of Program Evaluation and included four levels of evaluation: (1) reaction of program participants, (2) learning acquired by program participants, (3) behavior changes in program participants, and (4) overall results of the program on the participants and benefits to the school district. The study was guided by the primary research question: How effective is a school system’s research-based Grow Your Own leadership development program in growing a pool of qualified candidates to fill future leadership positions? Secondary questions were: (1) Did the Grow Your Own leadership program meet the needs of the participants; (2) What did participants in the Grow Your Own leadership program learn relative to leadership principles; (3) How has the behavior and attitudes of the participants in the Grow Your Own leadership program changed relative to school leadership as a result of participating in the program; and (4) Did the Grow Your Own leadership program meet the goal of designing and implementing a program effective in increasing the pool of qualified and competent leaders in Craven County Schools? 143 The program evaluation study was divided into four phases of evaluation with questions aligned to Kirkpatrick’s (2006) model of evaluation. The phases of evaluation included (1) Monthly participant assessment of program components, (2) A summative questionnaire, (3) Evaluation of learning through a pre and post assessment of understanding of core leadership principles and (4) Interviews with selected participants to gain a deeper understanding of the short and long term benefits of the program. Kirkpatrick’s (2006) model was used to formulate central research questions. Under each central research question, sub-questions were used to gather data aligned with each level of Kirkpatrick’s (2006) model. The questions for level 1 (reaction) were designed to evaluate the participants’ reaction to the GROW program. Level 2 (learning) questions were developed to evaluate participants’ learning relative to the core leadership principles addressed in the GROW program. Questions in level 3 (behavior), were designed to evaluate the changes in behavior of the participants of the GROW program. Level 4 (results) questions were used to determine the results of the program and to determine the need for program improvements. Data used in the study was collected from the twenty four completers of the initial GROW program implemented in Craven County Schools. All GROW graduates were invited to participate in at least one phase of this research study. Of the participants, 100% were invited to participate in the monthly assessments, summative questionnaire, and the GROW evaluation (n=24). Forty-two percent of the GROW participants were invited to participate in individual interviews (n=10). The response rates for the phases in this study ranged from 88% (n=24) to 100% (n=10) and exceeded the target response rate of 85% set by the program evaluator. Chapter Four of this study included results of both quantitative and qualitative data used in this study. Quantitative data was analyzed and presented in numerical formats using tables 144 and figures. In order to gain more insight from participants regarding the GROW program, qualitative data was also collected. Analysis of qualitative data included coding and analysis of textual data in order to determine patterns and themes helpful in answering the research questions posed in this evaluation study. A summary of emergent themes in each evaluation level of Kirkpatrick’s model, along with overarching themes throughout the research was presented Findings Findings from this study suggest that the GROW program implemented in Craven County Schools was effective for participants and beneficial to the school district. These findings will provide district leaders in Craven County Schools information on the effectiveness of the Grow Your Own Leadership development program designed and implemented in the school district. In addition, the findings share the impact of the program on participants regarding their reaction to the program, behavior changes, learning, and results. Analysis of the data suggests the GROW program, based on perceptions of program participants, is effective in increasing the number of interested and qualified candidates for school leadership positions in Craven County Schools. Multiple data sources were used to gather both quantitative and qualitative data. The data was analyzed for common themes and patterns helpful in determining the success of the implementation of the GROW program. Analysis of the data used in this study revealed the following findings: Finding 1: Participants who completed the GROW program developed and implemented in Craven County Schools perceived to be better prepared for school leadership positions within 145 the district. Participants report program content and consultants contributed to their preparedness. Finding 2: GROW participants reported a positive reaction to the program and a high level of satisfaction with program content, activities, concepts, and presenters. Emergent themes relative to the components of the GROW program that were most positive for participants were self-reflection, collaboration, leadership, use of videos and tapping. Finding 3: The data supports a high level of learning relative to participant knowledge and understanding of the leadership principles as a result of participating in the GROW program. Emergent themes relative to participant learning in the GROW program were impact analysis, courage, decision making, and collaboration. Finding 4: An analysis of the data and emergent themes reported by participants suggests positive changes in the behaviors of those who participated in the GROW program. Emergent themes impacting behavior changes included an increased interest in leadership, more self- confidence, a realization of leadership potential, understanding the decision making process, developing strategies on how to build relationships, and knowing the territory in which they work. Finding 5: The research suggests a strong influence of tapping on the participants’ decision to apply for the GROW program. Of the program participants, 80% report being tapped by a principal or district leader. Colleague encouragement also impacted the application decisions of participants. Finding 6: Participants had positive perceptions regarding the impact of the cohort model used in the GROW program. The opportunity to collaborate with colleagues and the ongoing benefits of the relationships formed was determined to be a strength of the program. 146 Finding 7: Participants report the GROW program as being more relevant to learning about school leadership than university MSA programs. Participants report that the GROW program is more “real world” than the university programs in which they are enrolled. Finding 8: The data suggests positive results of the GROW program for both participants and the school district. Analysis of the coded responses of participants indicated emergent themes in courage, collaboration, and interest in leadership. Finding 9: The GROW program had a positive impact on participant interest in advancing to a leadership role. Based on an analysis of the responses, 95% (n=20) of GROW program participants reported an increase level of interest in school leadership due to program participation and reported that they were more likely to apply for leadership positions within the district since participating in the GROW program. Finding 10: All participants reported to believe the GROW program will contribute to a stronger pool of applicants for school leadership positions in Craven County Schools. Conclusions Multiple data sources and the analysis of both quantitative and qualitative data yielded significant findings in this study. Based on the findings, the following are conclusions relative to the GROW program implemented in Craven County Schools. Conclusion 1: The GROW program implemented in Craven County Schools was effective in increasing the pool of qualified candidate for school leadership positions. The program increased participant knowledge in core leadership principles and had a positive impact on the interest of participants in exploring opportunities in school leadership. Conclusion 2: Craven County Schools should continue to support and offer the GROW program for those interested in leadership development opportunities. 147 Conclusion 3: Craven County Schools should continue to partner with the Masonboro Group as consultants for program development and continuity. Participants reported the value of an outside perspective from those with a variety of leadership experiences. Conclusion 4: The GROW program should continue to use a cohort model as participants cited the benefits of developing a collaborative network of support. Conclusion 5: Partnerships should be developed further with university programs to ensure the complementary nature of district and university programs. Collaboration between universities and school districts has the potential to strengthen leadership development initiative at both sites. Recommendations for Programmatic Improvement and Support While program participants provided positive comments about the GROW program, content, and presenters, there were recommendation regarding program improvement initiatives that participants feel would strengthen program outcomes. Participants made the following recommendations relative to programmatic improvement and support: Recommendation 1: The GROW program should include a field-based learning component. Participants believe that there would be value in shadowing a school leader as part of the program requirements, as it would provide an authentic leadership experience. Participants suggest engaging in a field experience with an administrator outside of the building in which they currently work in order to gain a different perspective on leadership. Recommendation 2: One session of the GROW program should be designed to include those GROW graduates who are now in leadership positions as a resource for what the role is like on a daily basis. This interaction would provide practical advice from those who recently participated in the program and the impact of the program on their leadership practices. 148 Recommendation 3: GROW program graduates suggested having a follow-up session for all GROW I participants to network and discuss how the program has impacted their ability to lead in their current roles. Recommendation 4: An in-service should be conducted with current school administrators to share an overview of the content of the GROW program. Participants feel that this sharing would benefit the leaders and the district to ensure fidelity of implementation of program outcomes. In addition, participants believe current principals would benefit from program content. Recommendation 5: School leaders and GROW graduates should be encouraged to tap other potential leaders to apply and participate in the GROW program. Results of this study suggest that tapping would impact the interest level of potential candidates. There should be emphasis on increasing the number of minority candidates participating in the GROW program. Recommendation 6: To encourage graduates of the GROW program to seek principal licensure, GROW scholarships should be offered as an incentive for those interested in enrolling in an MSA program. Recommendation 7: GROW program outcomes should be shared with the Board of Education in order to maintain program support and justify the district’s return on investment. Recommendation 8: Grant funding should be secured to support the continuation and expansion of the GROW program and the implementation of other leadership development and support initiatives in Craven County Schools. Recommendation 9: Craven County Schools should continue to provide support to GROW graduates and school leaders by maintaining a district leadership coach. Participants 149 who have transitioned into leadership roles report the value of this mentoring relationship in contributing to their successful transition into leadership. Limitations This study was unique to one school district, therefore the total research population of this program evaluation was relatively small (n=24). A small sample makes it difficult to draw conclusions regarding long-term program impact or generalize findings to other school districts. Research participants were involved in the first cohort of the GROW program. In order to generalize the results, research should be ongoing and should include future GROW cohorts. Additional research with future GROW participants would provide a larger sample size and additional research data helpful in determining the long-term impact of the GROW program on participants and the school district. There were opportunities for bias to interfere with data collected for this study. First, indicators of program success relied heavily on participant perceptions. Perceptions can vary and may differ relative to the timing of the responses. There was no data collected that correlated participate perceptions with the perceptions of those leaders who serve as their supervisors. Another potential bias could be the perception of the participants that negative comments regarding the program could impact the opportunities for advancement into leadership positions. This response bias could have been present. Finally, the program evaluator in this study currently serves as an assistant superintendent in the study district. The supervisory role of the researcher has the potential to impact the responses of the participants due to the supervisory role of the evaluator. In order to minimize the likelihood that these potential biases impact the findings in this study, multiple sources of data were used and all sources except for individual participant interviews allowed for anonymity. 150 While it is important to reference possible limitations of this study, every effort was made by the program evaluator to minimize their potential impact on the study outcomes. The importance of the research and input from those who participated in the program outweigh any potential limitations of the study. Recommendations Further Research Based on the findings of the program evaluation study, the researcher suggests the need for continued research in order to guide future initiatives relative to the development of school leaders. The following are recommendation for further research: Recommendation 1: Conduct a longitudinal study over a five year period of GROW participants that have transitioned into leadership positions to examine the effectiveness of the GROW program in providing the skills necessary to be successful in the role of school leader. This study would provide data regarding the long-term impact of the program on participants and the school district. Recommendation 2: A study should be conducted to seek the perspectives of the leaders under which the GROW graduates work in order to gather data regarding the impact of the program on the leadership abilities and behavior changes of participants in the school setting. This study would allow for the triangulation of the data using sources not available in the current study. Recommendation 3: A study should be conducted to examine the perspectives of teachers with whom GROW graduates work in a school leadership position in order to determine the impact of the program on leading teachers and creating a culture teachers perceive as positive. 151 Recommendation 4: A program development study should be conducted relative to the elements necessary in establishing effective university-school district partnerships that would contribute to the development of school leaders and increase the number of qualified candidates for pending administrative vacancies. Concluding Remarks Effective school leaders are essential to school and district success, yet finding strong instructional leaders prepared to lead schools in the twenty-first century is more challenging than ever before (Whitaker, 2001). The challenge of finding qualified school leaders became very real for me as I transitioned into the role of Assistant Superintendent of Human Resource Services. In eighteen months alone, twenty-nine leadership positions were vacant in the district and few qualified candidates were interested in applying for the positions. In addition, the future looked even more challenging as 54% of current principals in the district would be eligible to retire in five or less years and 81% of principals would be eligible to retire in less than ten years. Our district was in crisis. Through the visionary leadership of a newly appointed superintendent, a Grow Your Own leadership development program was designed and implemented in the district. As with any venture, the focus is on results. That was the purpose of this program evaluation study; to determine if the program was successful in increasing the number of qualified and interested candidates in school leadership positions. This journey has been filled with enlightening realizations regarding leadership development. The study found the risk taken by Craven County Schools to develop an innovative way to address the administrative shortage was rewarded. The GROW program has reignited the passion in participants and we have more applications for leadership positions than I have seen in my three years as an assistant superintendent. In 152 addition, preliminary data from this study was used in a grant proposal to the Z. Smith Reynolds Foundation seeking funding support for leadership initiatives in Craven County Schools. It was exciting to learn, the grant was awarded in the amount of ninety thousand dollars. This makes the work that has gone into completing this study even more rewarding by knowing it will impact future leaders in our school district. The impression by those involved in the program is that the initiative is successful in developing leaders in the district. Positive feedback continues to be shared by GROW program graduates. Participants shared program value relative to leadership development. One participant stated, “This was the best professional development since beginning work in Craven County Schools. For me, it was money well spent. Administrators should be using these best practices when planning, presenting, and speaking at faculty meeting” (I-7). Another remarked, “The grow program provided the inspiration, education, and tools to lead in a thoughtful and systematic manner” (Q-20). Participants also report a renewed passion for education. A participant shared, “The program has lit a fire under me again” (Q-20). Another stated, “I became more passionate and excited about becoming a leader” (I-1). Finally, participants realize and appreciate the deep commitment of the school district and district leadership to provide leadership development opportunities for those who have the desire to impact change. One participant shared, “The GROW program is an amazing opportunity for Craven County Schools. I am just thankful I had the opportunity. The program gave me the confidence to lead” (I-6). The value of the program experience is revealed through the words of one of the participants, who stated, I've actually wanted to write the Board of Education a thank you note for beginning the GROW program. Sometimes teachers feel alone and that those above you don't 153 understand. The GROW program says to us, you are important. You are important enough for us to empower you to do something different. That made the program really special!” (I-8) School districts will continue to face the challenge of filling leadership positions with qualified candidates. Heath Morrison (2005) sums up the way in which school districts will have to address the challenge by stating, School systems will need to take a greater role in the identification and preparation of future administrative candidates. With increased expectations on student performance and accountability, future principals will need a blending of proven theoretical and pedagogical training, along with practical, hands-on leadership experiences that "grow your own" programs and universities can provide together. (p. 155) REFERENCES Ackoff, R. L. (1989). From data to wisdom. Journal of Applied Systems Analysis 15: 3-9. Adair, J. (2009). How to grow leaders: The seven key principles of effective leadership development. London, England: Kogan Page. Barner, R. (2006). Bench strength developing the depth and versatility of your organization's leadership talent. New York, NY: AMACOM/American Management Association. Barth, R. (2001). Learning by heart. San Francisco, CA: Jossey-Bass. Baruch, Y. & Holtom, B. C. (2008). Survey Response Rate Levels and Trends in Organizational Research. Human Relations 61(8), 1139–1160. Beach, G. (2010). Conditions affecting the decision to seek or not seek a position as a school assistant principal/principal. (Doctoral dissertation), Available from ProQuest. (AAT 3408999)Retrieved from http://proquest.umi.com/pqdweb?did=2114596511&Fmt=2&clientId=48948&RQT=309 &VName=PQD&cfc=1 Branch, G., Hanushek, E., & Rivkin, S. (2012). Estimating the effect of leaders on public sector productivity: The case of school principals. Working Paper No. 17803. Cambridge, MA: National Bureau of Economic Research. Brousselle, A., & Champagne, F. (2011). Program theory evaluation: Logic analysis. Evaluation and Program Planning, 34(1), 69-78. doi: 10.1016/j.evalprogplan.2010.04.001 Browne-Ferrigno, T., & Muth, R. (2004). Leadership mentoring in clinical practice: Role socialization, professional development, and capacity building. Educational Administration Quarterly, 40(4), 468–494. 155 Bumphus, W. & Royal, A. (2008, October 24). Growing your own world-class leadership development institute. Innovative Abstracts, XXX(24), 1-2. Retrieved from http://www.scc.losrios.edu/Documents/StaffResourceCenter/NISOD/Vol. 30, No. 24.pdf Byham, W., Smith, A., & Paese, M. (2002). Grow Your Own leaders: How to identify, develop, and retain leadership talent. Upper Saddle River, NJ: Prentice-Hall, Inc. Campbell, D. F. (2002). The leadership gap: Model strategies for leadership development. Washington, DC: Community College Press, American Association of Community Colleges. Canavan, K. (2001) Leadership Succession in Catholic Schools: Planned or Unplanned. Catholic Education: A Journal of Inquiry and Practice, 5 (1) 72- 84. Chain of Evidence. (n.d.). Chain of Evidence. Retrieved May 31, 2014, from http://www.kirkpatrickpartners.com/Home/KirkpatrickChainofEvidence/tabid/288/Defaul t.aspx Chapman, J. (2005). Recruitment, retention, and development of school principals. Paris: International Institute for Educational Planning (IIEP) Brussels. International Academy of Education (IAE). Cheney, G. R., Davis, J., Garrett, K., & Holleran, J. (2010). A new approach to principal preparation: Innovative programs share their practices and lessons learned. Fort Worth, TX: Rainwater Charitable Foundation. DOI: www.anewapproach.org Cibulka, J.G. (2004), “The case for academic program standards in educational administration: Toward a mature profession”, UCEA Review, Vol. 46 No. 2, pp. 1-5. Coelli, M., & Green, D. (2012). Leadership Effects: School Principals and Student Outcomes. Economics of Education Review, (31) 1, 92-109. 156 Cooley, V. E., & Shen, J., (1999, April). Who will lead? The top 10 factors that influence teachers moving into administration. NASSP Bulletin, 83(606), 75-80. Copeland, M. (2001). The myth of the superprincipal. Phi Delta Kappan, 8(7), 528-533. Cranston, N. (2007). Through the eyes of potential aspirants: Another view of the principalship. School Leadership & Management, 27, 109-128. Daresh, J. (2004). Mentoring school leaders: Professional promise or predictable problems? Educational Administration Quarterly, 40(4). Davis, S., Darling-Hammond, L., LaPointe, M., & Meyerson, D. (2005). School leadership study: Developing successful principals (Review of Research). Stanford, CA: Stanford University, Stanford Educational Leadership Institute. Dehart, A. (2011). A comparison of four frameworks of teacher leadership for model fit. (Doctoral dissertation). Retrieved from http://trace.tennessee.edu/cgi/viewcontent.cgi?article=2274&context=utk_graddiss Fitzpatrick, J. L., & Sanders, J. R. (2011). Program evaluation: alternative approaches and practical guidelines (4th ed.). Upper Saddle River, N.J.: Pearson Education. Fullan, M. (2001). The new meaning of educational change. (3 ed.). London: Teachers College Press. Fullan, M. (2001). Leading a culture of change. San Francisco, CA: Jossey-Bass. Fuller, E. & Young, M. (2009). Tenure and retention of newly hired principals in Texas. Texas High School Project. 1-19. Fulmer, R. M., & Goldsmith, M. (2001).The leadership investment: How the world's best organizations gain strategic advantage through leadership development. New York, NY: AMACOM. 157 Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction. New York: Longman Publishers. Grubb, W. & Flessa, J. (2006). A job too big for one: Multiple principals and other nontraditional approaches to school leadership. Educational Administration Quarterly, 42(4), 518-550. Grunow, A., Horng, E. H., & Loeb, S. (2010). Succession management in schools (Working paper). Stanford, CA: Stanford University, Center for Education Policy Analysis. Guskey, T. (2002). Does It Make a Difference? Evaluating Professional Development. Educational Leadership, 59(6), 45-51. Retrieved July 12, 2014, from http://www.ascd.org/publications/educational-leadership/mar02/vol59/num06/Does-It- Make-a-Difference’-Evaluating-Professional-Development.aspx Guterman, J. (2007, March 23). Where have all the principals gone?: The acute school-leader shortage. Edutopia. Retrieved from http://www.edutopia.org/where-have-all-principals-gone Gutmore, D. (2009). Meeting the needs: A best practice grow your own school leader program. AASA Journal of Scholarship and Practice, 6(1), 33-39. doi: http://www.aasa.org/jsp.aspx Hargreaves, A., & Fink, D. (2006) Sustainable Leadership, San Francisco, CA: Jossey-Bass. Harris, A. (2007). The current crisis in leadership: Threat or opportunity. School Leadership and Management, 27 (2), p.105-107. Hartle, F. & Thomas, K. (2003), Growing Tomorrow’s School Leaders: The Challenge. Nottingham: NCSL. 158 Hess, F., & Kelly, A. (2005). The accidental principal. Education Next, 5(3), pp. 35-40. Joseph, S. (2010). School district "grow your own" principal preparation programs: Effective elements and implications for graduate schools of education. International Journal of Educational Leadership Preparation, 5(2), doi: http://cnx.org/content/m34194/1.1/ Kiel, V. & Czerniak, C. (2003, October 1). The urban principal leadership program. Connections. Journal of Principal Preparation and Development, 5, 2-9. Retrieved from: http://www.principals.org/portals/0/content/48349.pdf Kelley, J. (2011). Teacher's and teacher leaders' perceptions of the formal role of teacher leadership. (Unpublished doctoral dissertation) Retrieved from http://digitalarchive.gsu.edu/cgi/viewcontent.cgi?article=1070&context=eps_diss Kesler, G. C. (2002, March 1). Why the Leadership Bench Never Gets Deeper: Ten Insights about Executive Talent Development. Human Resource Planning, 25, 32. KIPP: Visionary leadership. (2013, October 5). Retrieved from http://tntp.org/assets/tools/KIPP_Leadership_Framework_and_Competency_Model_FI NAL.pdf Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs the four levels (3rd ed.). San Francisco, CA: Berrett-Koehler. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2007). Implementing the four levels: a practical guide for effective evaluation of training programs. San Francisco: Berrett-Koehler Publishers. Lacey, K. (2003) Succession Planning in Education, Melbourne: Right Angles Consulting Pty Ltd. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.202.565&rep=rep1&type=pdf Lauder, A. (2000). The new look in principal preparation programs. NASSP Bulletin, 84, 23-28. 159 Lavinghouze, R.S., Price, A. W., & Smith, K. (2007). The program success story: A valuable tool for program evaluation. Health Promotion Practice. 8 (4), 323-331. Leithwood, K., Louis, K., Anderson, S., & Wahlstrom, K. (2004). How leadership influences student learning. Retrieved from http://www.wallacefoundation.org/knowledge- center/school-leadership/key-research/Pages/How-Leadership-Influences-Student- Learning.aspx Leskiw, S., & Singh, P. (2007). Leadership development: Learning from best practices. Leadership & Organization Development Journal, 28(5), 444-464. Leviss, K. (2011). The Perfect hire: A tactical guide to hiring, developing, and retaining top sales talent. S.l.: Entrepreneur Press. Lewkiw, S., & Singh, P. (2007). Leadership development: Learning from best practices. Leadership & Organization Development Journal, 28(5), 444-464. Mayfield, A. (2013, March 19). The Psychology of Survey Respondents. Retrieved February 2, 2015, from http://www.optimalworkshop.com/blog/the-psychology-of-survey- respondents/ Mezzacappa, D. (2008, May). A better pipeline to the principalship. Improving leadership for learning: Stories from the field. Center for the study of teaching and policy, University of Washington. Retrieved from http://www.wallacefoundation.org/KnowledgeCenter/KnowledgeTopics/CurrentAreasofF ocus/EducationLeadership/Pages/improving-leadership-for-learningstories-from-the- field.aspx. Morrison, H. (2005). A critical evaluation of a school system's effort to develop and implement a "grow your own" principal preparation program. (Doctoral dissertation). 160 Murphy, J. (2006) Preparing School Leaders: Defining a Research and Action Agenda. Lanham, MD: Rowman & Littlefield Education. New Leaders. (2011). New Leaders Urban Excellence Framework New York, NY: New Leaders, 2011). Retrieved from http://www.newleaders.org/wp- content/uploads/2011/08/UEF-ConceptMaps1.pdf. Parry, S.R. (1996) The Quest for Competencies, Training, July 1996, pp. 48-56. Patton, M. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, Calif.: Sage Publications. Race, P. (2002). Evidencing reflection: putting the 'w' into reflection.. Retrieved July 8, 2014, From http://www.heacademy.ac.uk/resources/detail/subjects/escalate/1049_Evidencing_ Reflection_putting Rhodes, C. & Brundrett, M. (2006). The identification, development, succession and retention of leadership talent in contextually different primary schools: a case study located within the English West Midlands. School Leadership and Management. 26, 269-287. Rogelberg, S. & Stanton, J. Understanding and dealing with organizational survey nonresponse. Organizational Research Methods, 2007, 10, 195–209. Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: a systematic approach (7th ed.). Thousand Oaks, CA: Sage Publications. Rothwell, W. (2001). Effective succession planning. New York, NY: American Management Association. Russon, C., & Reinelt, C. (2004). The results of an evaluation scan of 55 leadership development programs. Journal of Leadership & Organizational Studies, 10(3), 104-107. 161 Shulman, L. (2004). A new vision for the doctorate in education: Creating stewards of the discipline through the carnegie initiative on the doctorate. Symposium at the annual Meeting of the American Educational Research Association. San Diego, CA, April, 2004. Sousa, D. (2003). The leadership brain: How to lead today's schools more effectively. Thousand Oaks, CA: Corwin Press. Spencer, L. M., & Spencer, S. M. (1993). Competence at work: Models for superior performance. New York, NY: Wiley. Trammell, J. (2005). Learning about the learning center: Program evaluation for learning assistance programs. The Learning Assistance Review, 10(2), 31-40. Turnbull, B., Riley, D., Arcaira, E., Anderson, L., & MacFarlane, J. The Wallace Foundation, (2013). Six districts begin the principal pipeline initiative. Retrieved from Policy Studies Associates, Inc. website: http://www.wallacefoundation.org/knowledge-center/school- leadership/principal-training/Documents/Six-Districts-Begin-the-Principal-Pipeline- Initiative.pdf W. K. Kellogg Foundation. (2004). W. K. Kellogg Foundation logic model development guide. Retrieved from http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf Wallace Foundation. (2007). Getting principal mentoring right: lessons from the field. (A Wallace “Perspective” Report). Retrieved from http://www.wallacefoundation.org/KnowledgeCenter/KnowledgeTopics/CurrentA reasofFocus/EducationLeadership/Documents/Getting-Principal-Mentoring- Right.pdf 162 Wallace Foundation. (2012). The making of the principal: Five lessons in leadership training. (A Wallace “Perspective” Report). Retrieved from http://www.wallacefoundation.org/knowledge-center/school-leadership/effective- principal-leadership/Documents/The-Making-of-the-Principal-Five-Lessons-in- Leadership-Training.pdf Whitaker, K. (2001, May). Where are the principal candidates? Perceptions of superintendents. NASSP Bulletin, 85 (625), 82-92. Wilson, J. (2009). Administrator succession planning. The School Administrator, 66(11), 20-24. Winter, P., & Morgenthal, J. (2002). Principal recruitment in a reform environment: Effects of school achievement and school level on applicant attraction to the job. 38, 319-342. Witcher, R. (2011). A descriptive study of the factors that prevent principal candidates from advancement to the principal position. (Doctoral dissertation)Retrieved from http://scholarcommons.usf.edu/cgi/viewcontent.cgi?article=4607&context=etd Zenger, J. H., & Folkman, J. (2003). Developing leaders. Executive Excellence, 20 (9), 5. APPENDIX A: LETTER OF SUPPORT FROM DR. LANE B. MILLS, SUPERINTENDENT OF CRAVEN COUNTY SCHOOLS 164 APPENDIX B: VITAE OF PRIMARY CONSULTANT Edwin L. West, Jr. Post Office Box 7597 Wilmington, North Carolina 28406 Email: elwj@aol.com Education New Hanover High School, 1958 Wilmington College, A.A., 1960 University of North Carolina at Chapel Hill, B.S., 1962 University of London, Advanced Graduate Work, 1964-65 University of North Carolina at Chapel Hill, M.A., 1966 University of North Carolina at Chapel Hill, Ed.D., 1968 Additional American Management Association Professional Stanford University Training Vanderbilt University The Wharton School, University of Pennsylvania Union Theological Seminary, Richmond, Virginia Employment President, The Masonboro Group, Inc. Wilmington, North Carolina 1994-Present Superintendent, Gaston County Schools Gastonia, North Carolina 1990-1994 Superintendent, Pitt County Schools Greenville, North Carolina 1982-1990 Superintendent, High Point Public Schools High Point, North Carolina 1975-1982 Superintendent, Edenton-Chowan Schools Edenton, North Carolina 1972-1975 Director of Development, NC Department of Public Instruction 1970-1972 Science Consultant, NC Department of Public Instruction 1968-1970 Supervisor, Fifth Year Program University of North Carolina at Chapel Hill 1966-1968 Science Teacher, New Hanover High School Wilmington, North Carolina 1962-1964 and 1965-1966 166 Additional Vice-Chair, NC Center for the Advancement of Teaching Foundation Professional Board of Advisors, Belmont Abbey College Professional Past Chairman, North Carolina Annual Testing Commission Past Chairman, Board of Governors, Governor’s Schools of NC Professor, National Academy for School Executives Professor, National Academy for Vocational Education Adjunct Professor, University of North Carolina at Greensboro Adjunct Professor, East Carolina University Adjunct Professor, University of North Carolina at Chapel Hill Board of Directors, Public School Forum of North Carolina Board of Managers, North Carolina Congress of Parents &Teachers Professional American Association of School Administrators Affiliations North Carolina Association of School Administrators (Partial List) Association for Supervision and Curriculum Development National Art Education Association International Reading Association North Carolina School Public Relations Association Phi Delta Kappa Community Board of Directors, United Way Activities Board of Directors, YMCA (Partial List) Board of Directors, Arts Council Religious Sunday School Teacher, Masonboro Baptist Church Affiliations & Ordained Lay Minister, Presbyterian Church USA Activities (Partial List) Selected Honors (Partial List) Recipient, 1992, Superintendent of the Year North Carolina School Boards Association Recipient, 1992, Administrator of the Year North Carolina Association of Librarians Recipient, 1990, Executive Educator 100 Award as one of the one Hundred outstanding school executives in North America Recipient, 1987, John F. Kennedy Center for the Performing Arts, National Administrator of the Year Award Recipient, 1986, North Carolina Outstanding Administrator North Carolina Alliance for Arts Education Recipient, 1986, School Counselor Advocate of the Year North Carolina School Counselors’ Association Recipient, 1986, Administrator of the Year Award North Carolina Association of Educational Office Personnel Recipient, 1985, Friend of the Arts Award North Carolina Arts Education Association Recipient, Honorary Life Membership The North Carolina Congress of Parents and Teachers Recipient, 1973 Man of the Year Award Edenton, North Carolina 167 Publications “Teachers and Preachers: The Battle Over School Reform” (Partial List) Virginia Journal of Sociology, Summer, 1995 (with Kimon Sergeant). “We’ve Got Trouble…Right Here In…Unless…” National Art Education Advisory, Summer, 1993 (with Emmy C. Whitehead) “Redefining Accountability” The Tar Heel Administrator, Volume IV, Pages 8-10, May-June, 1992. “Schools, Colleges and Universities,” Vital Speeches of the Day, Volume VVII, No. 8, Pages 241-243, February 1, 1992. Empowerment in Educational Organizations Prepared for the Center for Creative Leadership, Greensboro, North Carolina, 1990. “The Partner Principles,” The National Educational Secretary, Volume 50, No. 1, Fall, 1984 Activities to Grow On, Volume 1-2, White Plains, New York: Grolier Publishing Company, 1982 (with Robert N. Fortenberry and Swinton Hill). “An Alternative Learning School: Student Results and System Consideration,” Journal of Instructional Psychology, Volume 9, Pages 122-132, 1982 (with D.H. Reilly and J. Reilly). Foundations of Reading, Volumes 1-9, White Plains, New York: Grolier Publishing Company, 1981 (with Robert N. Fortenberry and Swinton Hill). Basic Expectations, Stanford, Connecticut: Innovative Sciences, Inc.,1981 (with Robert N. Fortenberry and Swinton Hill) Leadership Development, Prepared for the National Center for Research in Vocational Education, Columbus, Ohio, 1979 (with L. Linton Deck, Robert N. Fortenberry and Pamela S. Mayer). School Community Relations: A Practical Approach. Lexington, Massachusetts: Xerox Publishing Company, 1978.“The Cool Connection: A Middle School Alternative.” The Middle School Journal, Volume 9, Number 4, November, 1978 (with Larry Allred and Paul B. Hounshell). Presentations American Association of School Administrators (Partial List) American Institute of Architects Center for Applied Technology IBM Executive Conference National Art Education Association North Carolina Association of Music Education North Carolina Congress of Parents and Teachers North Carolina Art Education Associations North Carolina Association of Educators of Young Children North Carolina Guidance Counselors Association North Carolina Science Teachers Association North Carolina Association of School Librarians North Carolina School Boards Association 168 APPENDIX C: COMPETENCY FRAMEWORK APPENDIX D: BROCHURE 170 APPENDIX E: EMAIL CALLING FOR APPLICATIONS Teacher Leadership Program Call for Applications Do you want to be part of something special? Do you have an interest in school leadership? Do you know a great teacher who should consider becoming a school leader? If so, consider applying or recommending applicants for the 1st Class of GROW Participants. Introducing the 2013–14 Teacher Leadership Program (GROW) Craven County Schools is pleased to announce that it is accepting applications for its 2013-2014 Teacher Leadership Program (GROW). The purpose of the Program is to cultivate the next generation of school leaders. The program identifies, recruits, and trains educators who share Craven County School System’s commitment to making a positive difference in education through positive leadership practices. The program offers the opportunity to develop those interested in school leadership positions while offering networking opportunities with other educators. Requirements & Expectations Craven County School System is seeking the best and brightest educators to help lead our schools to performance excellence. Applicants should have at least 3 years of successful teaching experience and an interested in future leadership opportunities. Applicants who are selected to participate will be expected to attend all program events. This program will seek to identify and recruit the best teachers to become the best leaders. Please review the enclosed brochure for more information. The application is also attached. The deadline to apply is August 2, 2013. If you have questions or would like more information about the program, please feel free to contact me. Thanks, Wendy A. Miller Assistant Superintendent Human Resource Services Craven County Schools 252-514-6367 APPENDIX F: GROW APPLICATION Directions: Please complete each of the following spaces/ answer each question. Name: ______________________________ Position:______________________ School: __________________________________________________Date:_____ 1. Reasons for pursing a position in the Teacher Leader Program. Please check one. ____ Interest in School Administration ____ Professional Growth ____ Desire to become a Teacher Leader in my school ____ Other (Please specify) 2. I have been teaching for: ____ 3-5 Years ____ 11-15 Years ____ 6-10 Years ____ 16+ 3. Degrees Held/Certifications held: ____B.A/B.S. ____M. A./M.S. ____ Ed.D/PhD. ____ National Board Certification ____Other (Please List) __________________________________________________ 4. Please list the leadership role(s), if any, you have held in your school. a. _________________________________ c. ____________________________ b. _________________________________ d. ____________________________ Teacher Leadership Program (GROW) Application 173 5. Please concisely describe what you have learned from serving in this(ese) leadership roles. ____________________________________________________________________ ____________________________________________________________________ 6. Are you willing to take several assessment instruments that will help you assess your leadership skills, strengths and areas for improvement? _____ Yes _____ No 7. Please concisely describe what you believe to be your strengths and weaknesses as a leader. Strengths Weaknesses 1. ___________________________________ 1. _____________________________ 2. ___________________________________ 2. _____________________________ 3. ___________________________________ 3. _____________________________ 4. ___________________________________ 4. _____________________________ 5. ___________________________________ 5. _____________________________ 8. Would your principal agree with your self-assessment? _____ Yes _____ No _____Not Sure 9. The Teacher-Leader Program will be very intensive and require professional study over and beyond the regular school day. Are you willing to commit to fulfilling the time required to fulfill program requirements such as group projects, etc.? _____ Yes _____ No _____ Not Sure 10. Do you think your principal would recommend you for this program. _____ Yes _____ No ____ Not Sure APPENDIX G: GROW ROSTER Craven County Schools 2013-2014 GROW Program Roster PARTICIPANT Race Gender School Position Experience (Years) Degree Held NBCT P 1 B/F New Bern High Math Teacher 6-10 BA/BS P 2 W/F Creekside Elementary 4th Grade Teacher 11-15 MA/MS P 3 W/F Bridgeton Elementary 2nd Grade Teacher 16+ MA/MS P 4 W/M West Craven High English Teacher 3-5 MA/MS P 5 B/M Havelock Middle Administrative Assistant 11-15 BA/BS P 6 W/F Central Services STEM Project Manager 6-10 MA/MS P 7 W/F Central Services STEM Project Manager 16+ MA/MS P 8 W/F Vanceboro /Brinson Memorial Curriculum Coach 11-15 BA/BS P 9 W/F Early College East College Readiness Coordinator 11-15 MA/MS P 10 W/F H. J. MacDonald Middle 6th Grade Math/Sci Teacher 3-5 MA/MS P 11 W/F James W. Smith Elementary Kindergarten Teacher 6-10 BA/BS P 12 W/F A. H. Bangert Elementary 5th Grade ELA/SS Teacher 16+ BA/BS ? P 13 W/F J. T. Barber Elementary EC Resource Teacher 6-10 BA/BS P 14 W/F J. T. Barber Elementary 2nd Grade Teacher 11-15 BA/BS ? P 15 W/M West Craven High CTE Teacher (Agriculture) 3-5 BA/BS P 16 W/M James W. Smith Elementary 2nd Grade Teacher 3-5 BA/BS P 17 W/F West Craven Middle Physical Education Teacher 3-5 BA/BS P 18 B/M Trent Park Elementary Music Teacher 6-10 BA/BS ? P19 W/F Brinson Memorial Elementary 3rd Grade Teacher 16+ MA/MS P20 W/F Trent Park Elementary 5th Grade Teacher 11-15 BA/BS P 21 W/F Bridgeton Elementary Title I Teacher/Interventionist 16+ MA/MS P 22 W/F Tucker Creek Middle Media Coordinator 11-15 MA/MS P 23 W/F Grover C. Fields Middle 7th Grade Science Teacher 6-10 BA/BS P 24 W/F H. J. MacDonald Middle 8th Grade Math Teacher 3-5 BA/BS P 25 W/M Oaks Road Elementary 5th Grade Teacher 3-5 BA/BS APPENDIX H: GROW ACCEPTANCE LETTER September 20, 2013 Dear Applicant, Congratulations! It is a great pleasure that I inform you that you have been selected to participate in the Teacher Leadership Program (GROW). The selection committee reviewed over 60 applications for the 25 available spaces in the program. Your inclusion in this group is a testament to your hard work and your commitment to leadership that distinguishes you from your peers. You are to be congratulated on your acceptance in this very competitive process. I have enclosed information regarding dates of the sessions. All sessions will occur during the day. The expectation is that those selected to participate in the program attend all GROW sessions. For the sessions that are on school days, substitutes will be provided using program funds. Your acceptance is quite an accomplishment, but requires commitment on your part. Included with the letter is your intent to participate in the program. We need this back as soon as possible since there is a waiting list should seats become available. Return of forms electronically is preferred. In selecting you to participate in the GROW Program, we express profound admiration for your accomplishments to date and enormous confidence in your potential for future leadership opportunities. It is my hope that you will accept this offer to “grow” as a learner and a leader in Craven County Schools. Feel free to contact me with questions. Sincerely, Wendy A. Miller Assistant Superintendent Human Resource Services Craven County Schools APPENDIX I: GROW INTENT TO PARTICIPATE Name: _______________________________________ Position:______________________ School: _____________________________________________________________________ Session Dates: • October 14, 2013 (“O” Day) • November 8, 2013 (“S” Day) • December 11, 2013 • February 17, 2014 (“O” Day) • March 17, 2013 (“O” Day) • April 25, 2014 • May 12, 2014 • June 10, 2014 (“O” Day) Intent to Participate: I understand that I have been selected to participate in the Teacher Leadership Program (GROW) sponsored by Craven County Schools. I intend to participate and commit the time and effort required to benefit fully from the program. I understand that I am expected to attend all sessions and participate in activities in order to “grow” as a leader in my profession. __________________________________________________ _____________________ Signature Date Teacher Leadership Program (GROW) Intent to Participate APPENDIX J: INFORMED CONSENT TO PARTICIPATE IN RESEARCH East Carolina University Informed Consent to Participate in Research Information to consider before taking part in research that has no more than minimal risk. Title of Research Study: An evaluation of a School System's Effort to Develop and Implement a Leadership Preparation Program Principal Investigator: Wendy A. Miller Institution/Department or Division: East Carolina University Address: 1001 East 5th Street, Greenville, NC 27858 Telephone #: 252-328-6131 Study Sponsor/Funding Source: Craven County Schools Researchers at East Carolina University (ECU) studies problems in society, health problems, environmental problems, behavior problems and the human condition. Our goal is to try to find ways to improve the lives of you and others. To do this, we need the help of volunteers who are willing to take part in research. Why is this research being done? The purpose of this research is to evaluate the teacher leadership development program (GROW) implemented in Craven County Schools. The decision to take part in this research is yours to make. By doing this research, we hope to learn about and evaluate the success of the school system's efforts to develop and implement a grow your own leadership development program based on providing more qualified and interested candidates in school leadership positions The primary research question in this study is: • How effective is a school system’s research-based grow your own leadership development program in growing a pool of qualified candidates to fill future leadership positions? Why am I being invited to take part in this research? You are being invited to take part in this research because you participated in the GROW Program implemented in Craven County Schools. If you volunteer to take part in this research, you will be one of about 24 people to do so. Are there reasons I should not take part in this research? I understand I should not volunteer if I will not complete the 2014-2015 school year in Craven County Schools. What other choices do I have if I do not take part in this research? You can choose not to participate or to participate in only certain parts of this evaluation study. Page 1 of 3 Date:______________ ________________ Participant’s Initials 178 Where is the research going to take place and how long will it last? All research components will be conducted in Craven County Schools. The evaluation tools will be completed at Central Services and should take less than three hours to complete in the next two months. What will I be asked to do? You are being asked to do the following: • Give permission for the program evaluator to review the anonymous monthly assessments of the GROW program and use the data in the research. • Complete a summative evaluation of the GROW Program. • Complete a pre and post assessment of your understanding of the leadership principles addressed in the GROW Program. • Give permission for the program evaluator to review the reflective video developed in the GROW program and to use comments, video clips, and other data from the video in this research study. • Participate in an audio recorded individual interview (if selected) with the program evaluator relative to the potential short and long term benefits of the program. What possible harms or discomforts might I experience if I take part in the research? It has been determined that the risks associated with this research are no more than what you would experience in everyday life. What are the possible benefits I may experience from taking part in this research? We do not know if you will get any benefits by taking part in this study. This research might help us learn more about the impact of the GROW Program and ways to improve the program for future participants. There may be no personal benefit from your participation but the information gained by doing this research may help others in the future. Will I be paid for taking part in this research? You will not be compensated for your participation in this study. What will it cost me to take part in this research? It will not cost you any money to be part of the research. There is no cost associated with the collection of data used in this research. Who will know that I took part in this research and learn personal information about me? To do this research, ECU and the people and organizations listed below may know that you took part in this research and may see information about you that is normally kept private. With your permission, these people may use your private information to do this research: • Any agency of the federal, state, or local government that regulates human research. This includes the Department of Health and Human Services (DHHS), the North Carolina Department of Health, and the Office for Human Research Protections. • The University and the Institutional Review Board (IRB) and its staff, who have responsibility for overseeing your welfare during this research, and other ECU staff who oversee this research. Page 2 of 3 Date:______________ ________________ Participant’s Initials 179 How will you keep the information you collect about me secure? How long will you keep it? All data collected (physical and electronic) will be stored in a secure location in the researcher’s office during the study. The data collected will be used for this study and may be presented to the superintendent and Craven County Board of Education. The information collected will be stripped of identifiers and will be described as individuals using a letter of the alphabet in the reporting by the researcher. The only possible identifying data will be video clips approved for use by program participants. The data will be destroyed after five years following the completion of the research to ensure confidentiality. What if I decide I do not want to continue in this research? If you decide you no longer want to be in this research after it has already started, you may stop at any time. You will not be penalized or criticized for stopping. You will not lose any benefits that you should normally receive. Who should I contact if I have questions? The researcher conducting this study will be available to answer any questions concerning this research, now or in the future. You may contact the Principal Investigator at 252-670-7170 (Monday thru Friday, 8:00 am- 5:00 pm). If you have questions about your rights as someone taking part in research, you may call the Office of Research Integrity & Compliance (ORIC) at phone number 252-744-2914 (Monday thru Friday, 8:00 am- 5:00 pm). If you would like to report a complaint or concern about this research study, you may call the Director of the ORIC, at 252-744-1971. I have decided I want to take part in this research. What should I do now? The person obtaining informed consent will ask you to read the following and if you agree, you should sign this form: • I have read (or had read to me) all of the above information. • I have had an opportunity to ask questions about things in this research I did not understand and have received satisfactory answers. • I know that I can stop taking part in this study at any time. • By signing this informed consent form, I am not giving up any of my rights. • I have been given a copy of this consent document, and it is mine to keep. _____ Participant's Name (PRINT) Signature Date Person Obtaining Informed Consent: I have conducted the initial informed consent process. I have orally reviewed the contents of the consent document with the person who has signed above, and answered all of the person’s questions about the research. _____ Person Obtaining Consent (PRINT) Signature Date Page 3 of 3 Date:______________ ________________ Participant’s Initials APPENDIX K: MONTHLY ASSESSMENT SURVEY GROW Session Evaluation Teacher Leader Program Directions: Please share your thoughts for today’s session on Leadership Principles circling your response using the following scale: Y= yes N= No. Thank you for your kind assistance. 1. The objectives of the session were clear. Y N 2. The learning activities enhanced my Y N understanding of concepts. 3. The sequence of the session was appropriate. Y N 4. I gained awareness of some new concepts. Y N 5. The information learned will assist me in my Y N current and future work. 6. The handouts were helpful. Y N Comments (optional): APPENDIX L: GROW SUMMATIVE QUESTIONNAIRE Grow Questionnaire Directions: Please assist the Human Resources Department in reviewing the GROW program by responding to the following questions. Your thoughtful assistance in greatly appreciated. 1. Did program participation increase your interest in advancing to a leadership role? Yes No If yes, please state why. If no, why not? 2. Rate your intent relative to your interest in applying for a school leadership position within the next five years. More Likely Less Likely No Change 3. How did your participation in the program impact your desire to become a school leader? More Likely Less Likely No Change 4. Do you feel the GROW Program will help develop a stronger pool of applicants for school leadership positions? Yes No If yes, please state why. If no, why not? 182 5. How has the GROW program shaped your own beliefs about education and school leadership? What aspect(s) of the program most influenced your beliefs? 6. Overall, what does the GROW program do most effectively? What parts of the program have transformed you most as a leader? 7. Overall, how well prepared do you think you will be at the end of the program to become a leader in your school? 8. If you had to include additional experiences in the GROW program in order to better prepare participants for school leadership positions, what would you included? APPENDIX M: GROW EVALUATION GROW Evaluation Directions: During this school year, you were selected to participate in the GROW Leadership Program. Listed below are the core leadership principles that were the focal points for the year. Please take a few minutes to reflect on each principle. Please place a circle around the number that reflects where you think you were regarding your understanding of these leadership principles at the beginning of the GROW program. Please place a square around the number that delineates where you think you currently are in your understanding of these leadership principles. Please return this questionnaire to Ms. Wendy Miller. Thank you for your thoughtful and kind assistance. Principle Self Rating Know Thyself 1 2 3 4 5 6 7 Core Values 1 2 3 4 5 6 7 Courage 1 2 3 4 5 6 7 Know the Territory 1 2 3 4 5 6 7 Build a Culture 1 2 3 4 5 6 7 Goal Setting 1 2 3 4 5 6 7 Communication 1 2 3 4 5 6 7 Decision Making 1 2 3 4 5 6 7 Impact Analysis 1 2 3 4 5 6 7 Conflict Management 1 2 3 4 5 6 7 Accountability 1 2 3 4 5 6 7 Comments (optional): 184 APPENDIX N: GROW PARTICIPANT INTERVIEW PROTOCOL GROW Program Participant Interview Protocol and Interview Note Sheet Participant Number_________ M or F Ethnicity ____________ Interview Date__________ Years in Education_____________ Current Position________________________________ Level l Questions (Reaction): 1. Why did you initially apply for the GROW Program? Did someone encourage you to apply? If so, did that impact your decision to apply? 2. Tell me about your overall program experience in the GROW Program: 3. What aspect of the program was most valuable to you? 4. Least valuable? 5. Do you feel the information learned in the GROW Program will assist with your current and future work? 6. If you had to do it all over again, would you participate in the GROW Program? Why or why not? Level 2 Questions (Learning): 7. What leadership principles do you feel you most developed while participating in the GROW Program? 8. Do you believe the GROW Program will help develop a stronger pool of applicants for school leadership positions. Why or why not? 185 9. As part of the program, you learned about your leadership style. Share with me what you learned about yourself as a leader and how this has impacted how you lead others. 10. Describe the learning that took place for you as a result of participating in the GROW Program? Level 3 Questions (Behavior): 11. How have you applied what you learned in the GROW Program to your daily work? 12. What aspects of the program most influenced you as a leader? 13. What topics in the program changed your behavior as a leader? Why? 14. Before beginning the program, you were asked about your interest in leadership. Has that change? 15. Are you more or less interested in becoming a school leader since participating in the program? What part of the program most impacted this decision? 16. What are your future plans relative to school leadership? Level 4 Questions (Results): 17. Do you feel the investment the district made in the GROW Program was money well spent? Why or why not? 18. What would you identify as the most effective elements of the GROW program? 19. What are you doing differently since participating in the GROW Program? 20. Are you more or less likely to apply for leadership positions within the district since participating in the GROW Program? Explain your response. 186 21. Have you transitioned into a new leadership role since completing the GROW Program? 22. What other benefits to you, those you lead, and to the district were realized as a result of the GROW Program? 23. What can be done to enhance the quality of the program for future participants? 24. What additional supports would you recommend the district put in place to further assist in leadership development in Craven County Schools? 25. Did you encourage others in your school to apply for the GROW Program? Did he/she apply? Additional Comments: APPENDIX O: IRB APPROVAL EAST CAROLINA UNIVERSITY University & Medical Center Institutional Review Board Office 4N-70 Brody Medical Sciences Building· Mail Stop 682 600 Moye Boulevard · Greenville, NC 27834 Office 252-744-2914 · Fax 252-744-2284 · www.ecu.edu/irb Notification of Initial Approval: Expedited From: Social/Behavioral IRB To: Wendy Miller CC: Jim McDowelle Date: 1/5/2015 Re: UMCIRB 14-001951 AN EVALUATION OF A SCHOOL SYSTEM'S EFFORT TO DEVELOP AND IMPLEMENT A LEADERSHIP PREPARATION PROGRAM I am pleased to inform you that your Expedited Application was approved. Approval of the study and any consent form(s) is for the period of 1/4/2015 to 1/3/2016. The research study is eligible for review under expedited category #6, 7. The Chairperson (or designee) deemed this study no more than minimal risk. Changes to this approved research may not be initiated without UMCIRB review except when necessary to eliminate an apparent immediate hazard to the participant. All unanticipated problems involving risks to participants and others must be promptly reported to the UMCIRB. The investigator must submit a continuing review/closure application to the UMCIRB prior to the date of study expiration. The Investigator must adhere to all reporting requirements for this study. Approved consent documents with the IRB approval date stamped on the document should be used to consent participants (consent documents with the IRB approval date stamp are found under the Documents tab in the study workspace). The approval includes the following items: Name Description Dissertation Proposal.pdf Study Protocol or Grant Application GROW EVALUATION.pdf Surveys and Questionnaires GROW PARTICIPANT INTERVIEW PROTOCOL.pdf Interview/Focus Group Scripts/Questions GROW SUMMATIVE QUESTIONNAIRE.pdf Surveys and Questionnaires INFORMED CONSENT TO PARTICIPATE IN RESEARCH.pdf Consent Forms MONTHLY ASSESSMENT SURVEY.pdf Surveys and Questionnaires The Chairperson (or designee) does not have a potential for conflict of interest on this study. IRB00000705 East Carolina U IRB #1 (Biomedical) IORG0000418 IRB00003781 East Carolina U IRB #2 (Behavioral/SS) IORG0000418