PBT STEPS facilitates a learning experience centered around students’ development of deliverables for PHAs through: 1) information dissemination during in-class lectures, required and suggested readings, and guest speakers; 2) class exercises and case studies that facilitate critical thinking and skill application; 3) real-time discussions between stakeholders to address issues and questions; 4) opportunities for interactions with the PHA and community; and, 5) self-reflection and strategic feedback mechanisms to enhance real-time engagement. Course deliverables are designed to demonstrate workforce skills that align with course competencies and respond to the PHA’s priority issues ( Table 1 ).

¹The Council on Education for Public Health (CEPH) recognizes 8 fundamental MPH core competencies with a subset of 21 competencies. Each workforce skill includes its related fundamental competency in bold. Source: MPH Foundational Competencies, https://media.ceph.org/documents/D2_guidance.pdf . ²These objectives are created based on the Intervention Mapping framework. Source: Bartholomew LK, Parcel GS, Kok G. Intervention mapping: A process for developing theory and evidence-based health education programs. Health Education & Behavior. 1998; 25(5): 545-63. https://doi.org/10.1177/109019819802500502 .

Step 1: Securing Partnerships

A reciprocal collaboration for a successful PBT partnership necessitates the PHA’s issue area to align with course competencies. In return, the time investment from the PHA results in products tailored to their community. One month before the semester, partners set clear expectations regarding time commitment, engagement, and communication. These expectations are reiterated in writing (i.e., syllabus, detailed assignment guidelines, course timeline). Faculty can find collaborating PHAs through SPH connections, alums, previous PHA referrals, and colleague referrals.

The partnership with the Agency for the Fall 2021 semester was established through an initial meeting at a national public health conference the year prior. Fall 2021 was not the first time the Agency participated in the course, but conversations about the new collaboration began in early Summer 2021 to address ideas for the Agency’s issue area, due dates of deliverables and meetings schedules, the Agency’s competing priorities, and community stakeholders for engagement throughout the semester.

The priority issue for the Agency was community trauma among its residents. Numerous communities faced the impacts of community trauma stemming from the COVID-19 pandemic and the recognition of long-standing systemic racism. In particular, the community served by the Agency reeled from the effects of a fatal hate crime in early 2021 that garnered national attention. While the Agency worked with local organizations to respond to the impacts of trauma, they recognized the need to build capacity and expand resources through external partnerships and community-engaged approaches. The charge to the students working with the Agency in Fall 2021 was to respond to the impacts of trauma manifesting in the community with an intervention designed to engage youth.

Step 2: Technology and Training

The PBT framework facilitates seamless collaboration with PHAs regardless of where they are geographically located. Effective PBT courses use technology to streamline communication, encourage active engagement, and integrate robust learning management systems (LMS). This course uses a web-based experiential learning platform as an LMS and collaboration space. The platform allows access to course materials (i.e., readings, lectures, case studies) and the collaboration space encourages teamwork between PHAs, students, and faculty for real-time communication, document and feedback sharing, personal reflection, and peer review. Using technology, students also learn to explore creative solutions given limited resources. PBT classrooms can integrate expert guest lecturers to expose students and PHAs to often freely-available digital health and web-based platforms that support intervention implementation and communication.

The student team collaborated virtually with the Agency throughout the semester, increasing meeting flexibility while reducing travel time and exposing students to professionalism in a hybrid environment. Discussion boards were used throughout the semester for students to pitch ideas, the Agency to provide feedback on the student work products and interactions, and to document the process of deliverable development. The regularly scheduled virtual meetings ensured final intervention deliverables were feasible and tailored to the community. The final deliverables for the community trauma project utilized various technologies such as video, audio, and visual editor software, wireframing, and product design tools. In addition to accessing technology for virtual collaboration, exposure to new technologies strengthened students’ technical prowess while illustrating the potential for technology to enhance current and future programming in the Agency.

Step 3: Engagement and Implementation

Consistent communication among faculty, students, and PHAs, synergy within student teams, and ongoing opportunities for feedback are essential for the successful implementation of PBT courses. To support this, expectations reflect the demands of the current public health landscape by prioritizing timeliness, cultural responsiveness, and ingenuity. Student teams, typically representing various public health disciplines within the SPPH, as well as diverse experiences and student status (i.e., part-time vs. full-time), create team agreements that resemble a work contract. These agreements specify group expectations and rules of engagement. Early course lectures and discussions cover professionalism and consultation techniques to support appropriate engagement with the PHA. Together this facilitates the development of harmonious deliverables representing the teams’ strengths. Throughout the course, the teaching team compiles feedback from the PHA and the students to inform real-time adjustments to the pedagogy. This approach supports the teaching team in facilitating the needs of both the students and PHAs in a fast-paced learning environment. Subsequently, the teaching team conducts a comprehensive review of the feedback post-course to implement further improvements for future semesters.

The community trauma team communicated with identified community members through email, virtual, and in-person discussions. Members also conducted community site visits to assess the Agency’s context of implementation. This level of engagement gave depth to the intervention and communication strategy made possible by the PBT course design. Each of the three course deliverables had multiple opportunities for feedback. Consequently, the final assignment for the Agency presented clear intervention ideas and specific strategies to address community trauma.

After each assignment, the community trauma team received feedback via a standardized form from the Agency on the deliverables’ quality, the utility in practice, and the level to which it met field expectations. The feedback received from the Agency on the community trauma team project indicated that student assignments “met” the client’s expectations. Students respond to the same question prompt when submitting the assignment, with student self-reported responses indicating they felt the deliverables “exceeded” the Agency’s expectations. This discrepancy highlights an important point to students, signaling them to the high demands and expectations of the field. By integrating the Agency’s feedback into the development of deliverables, the PBT framework ensures that the final intervention is both relevant and impactful, aligning with the Agency’s needs and encouraging students’ abilities. Students also submitted self-reflections on their performance and that of their peers. The teaching team aggregated this feedback and reported back to student groups to offer areas for improvement and validation of successful collaboration processes.

Step 4: Presenting Deliverables

PBT courses ensure all assignments act as deliverables for the PHA to use in the field. Deliverables must be in formats that succinctly represents the shared vision of the intervention plan and communication strategy. Oral presentations of the final interventions include a question-and-answer session where students offer insight into the products’ implementation. During this time, students demonstrate more advanced deliverables, including mobile apps, videos, website designs, and other technologies, and describe traditional written project proposals or grant applications. The presentation with the PHA allows students to practice professional presentation skills, which are essential in today’s workforce.

The community trauma team designed an evidence-informed youth leadership and advocacy curriculum. The curriculum included newly developed resources and publicly available materials. It outlined daily objectives and activities to engage youth in anti-racist activism, trauma-informed strategies, and leadership skill building [30] - [32] . The curriculum intentionally engaged multiple community stakeholders to promote healthy youth-adult relationships, offer mentorship, and elevate youth power within the community. The final proposal included a budget, a three-year project timeline, a logic model, and a strategic communication plan. The communication plan addressed participant recruitment, appealed to funders and adopters, established community trauma norms, promoted the program to the community, and fostered interagency collaboration.

Step 5: Sizing Up Results

Continual evaluation of the pedagogy is important for resource allocation and curriculum improvement [20] . Teaching evaluations are not enough to provide the whole picture of the effectiveness of PBT; other assessment methods are essential [33] - [35] . Rarely, are the experiences of the PHA assessed, the application of student skills longitudinally tracked, and utility of practice products examined post-semester. The following data presents aggregated findings for the outcomes of the Fall 2021 semesters for all students. This data was collected via anonymous survey and cannot be stratified by student groups.

The Fall 2021 semester was evaluated using a variety of methods that went beyond the usual student course evaluations to examine the effectiveness of outcomes for all PBT stakeholders. This was part of a larger semester-long evaluation with the results specific to this case study highlighted here and results of the pedagogy, including a mixed-methods cross-discipline evaluation, published elsewhere [20] . Outcomes were assessed according to the course logic model and an independent evaluator conducted the evaluation [20] . The University’s Institutional Review Board (IRB Number #H-37484) approved all activities as an exempt study. The evaluation of student learning included: 1) course documents to provide course context and develop surveys; 2) post-assignment peer evaluations to assess the quality of group interactions; 3) pre-course (first week of class) and post-course (last week of class) surveys to assess achievement of competencies, satisfaction, and perceptions of PBT using a 5-point Likert Scale (strongly agree to strongly disagree or extremely important to not important); 4) grading rubrics to determine the range of grades in the semester; and, 5) the SPH student course evaluations to evaluate student reflections on the course, instructor, assignments, and activities. Data was analyzed according to qualitative and quantitative methods used in other PBT studies [20] .

The students enrolled in the Fall 2021 semester (n = 24) completed a pre-course (n = 22) and post-course (n = 19) survey with 18 surveys matched at pre-test and post-test. Of the students responding to the pre-course survey, most respondents were female (90.91%, n = 20), aged 23 - 25 (54.55%, n = 12), and were full-time students (50%, n = 11) with minimal work experience (data not shown). Students reported a significant increase in the acquisition of skills that were the focus of the course from pre-course to post-course in all but one of the skill areas (giving presentations). The most significant skill increase occurred in consultation techniques and the primary competency areas of the course ( Table 2 ). Additionally, post-course, students agreed that the time invested in taking a PBT course was worth it, more MPH courses should use PBT, and they appreciated the utility of the PBT course more post-semester than when they had initially enrolled (78%, n = 14) ( Table 2 ). Students disagreed that they would have gained the same knowledge (83.33%) and the same skills (94.54%) if the course used traditional teaching methods rather than PBT (data not shown). Some skills were not significantly different with some reported slight decreases post-semester. This could be due to a ceiling effect with high pre-test scores, the pedagogical approach not impacting those skills, and/or perception of ability of those skills not fully realized at post-course [20] .

Students reported that working with a PHA in a PBT course impacted their workforce readiness ( Table 2 ) specifically by contributing to securing a position in the field and that PBT made them more marketable job candidates (4.37, std = 0.60), better prepared them to enter the workforce (4.00, std = 0.58), and enhanced their appreciation of public health practice (4.37, std = 0.76). In addition, students consistently agreed that PBT helped with skill development that is essential in the workforce, including problem-solving skills (4.21, std = 0.54), leadership skills (4.10, std = 0.74), teamwork skills (4.32 (0.67), and skills they could implement in their job (4.58, std = 0.51).

The Agency also reported post-course utility of the assignments, quality of collaboration, and satisfaction with the PBT course through course feedback gathered through informal meetings and a course survey administered to the Agency. In Fall 2021 the Agency reported implementing the deliverables produced by the students. The Agency stated that the intervention proposals (a course

Footnotes: ¹Competencies listed map to the requirements of the course. ²Response options ranged from “no familiarity” = 1; “some ability” = 2; “average ability” = 3; “above average ability” = 4; “very able” = 5. ³p-values represent the comparison between pre/post are calculated at an alpha of 0.05 using the Wilcoxon Signed Rank Test.

deliverable) and collaboration/networking with faculty were the most beneficial aspects. Additionally, the Agency disagreed that it would be possible to develop the same deliverables without added resources or gain the same quality of deliverables if they had produced them internally. The Agency strongly agreed that the time invested in PBT was worth it and would consider additional PBT participation in the future (data not shown). The Agency stated that more MPH courses should utilize PBT.

In this case study, all stakeholders benefited from the collaboration beyond the PBT course. These benefits were uncovered through the evaluation process (public health agency), informal discussions (students and faculty), and formal interviews (public health agency, students, faculty). Benefits are described in detail below.

Through the formative evaluation, the Agency gained valuable insight into the most urgent needs of community members without utilizing their limited resources. The research helped the Agency refine its community trauma approach. Findings reinforced the need for a health equity framework to tackle the social factors contributing to the community’s current traumas and stigmas. Without the pre-established relationship with the PBT course, the Agency would have spent considerable resources conducting the same research. The successes of this research created opportunities for the Agency and the SPH to continue to conduct high-quality research with limited resources.

As a result of the collaboration, the Agency was well-positioned to apply for, and receive in the year following the course collaboration, funding that increased resources and capacity of the Agency to better serve the community. The Agency received several multi-year grants from the state to address the impacts of domestic violence, with a focus on trauma recovery, housing stabilization, and training of current public safety and public health practitioners. These grants also allocate funds for the hiring of new staff positions within the Agency. Additionally, a federal grant supports the integration of data across the community’s public health programs for real-time monitoring of trends. Collectively, these grants support sustainability and capacity within the Agency while strengthening connections among local agencies.

The Agency utilized a number of new tools through collaboration with the SPH, including written educational materials, research analysis software, and visualization tools that aid in communicating information to the community and beyond. The Agency and faculty disseminated their successful collaboration at multiple national conferences. This elevated the Agency’s national status as a community trauma response and health equity leader and the SPH as an academic institution committed to linkages with the community. In 2022, federal agencies recognized the Agency as a mentor site to other LHDs across the U.S. interested in multi-sector approaches to public health. In this role, which continues presently, the Agency provides technical assistance and support to four other states to foster cross-institutional collaborations to address pressing community issues.

The post-semester academic-community collaboration allowed the students working with the Agency to continue to refine their professional and research skills. Both students were fully engaged in the research process from Institutional Review Board (IRB) submission to the implementation of the mixed-method design. Guided by their own expertise and interests, the directed study student collected qualitative data through interviews, while the research fellow developed and analyzed quantitative surveys. Through peer collaboration, the students enhanced their respective skill sets. After some initial training, the faculty member supported the student researchers in leading the facilitation of interviews, completing qualitative data analysis through NVIVO coding sessions, aggregating the interview findings into recommendations for the Agency, and presenting the results to the Agency.

To support and sustain their research efforts post-PBT course, the directed study student received credit hours for the collaboration, and the research fellow received a standardized stipend through the school-based fellowship mechanism. Once the requirements of the directed study and fellowship ended, the students continued to work with the community-academic partnership in a volunteer capacity, given their investment in the work and motivation to see products from the collaboration. Faculty mentorship and partnership with the Agency enhanced students’ individual skill sets, making them more marketable to employers [11] [12] [41] and the deliverables served as writing samples for the students to secure jobs after graduation. Partially due to their ongoing collaboration with the Agency, both students acquired supervisory public health roles. Currently, a research student continues to work as an evaluation consultant for the Agency with two new students who joined the SPH team to continue working with the Agency.

While working with the Agency, the faculty benefited from a research environment that offered the potential for continued grant-funded opportunities. This is particularly important at the SPH, which relies on faculty securing their own funding. As an ongoing evaluator for the Agency, faculty built reputational capital for the SPH. The engagement of multiple students across the MPH program in this research advanced the faculty’s individual reputation as an effective academic-community collaborator and resource for the SPH.

Due to the partnerships’ success, the faculty was invited to present the PBT pedagogy at national academic and community institutions conferences. The presentation applied the PBT STEPS frameworks and highlighted the post-semester outcomes. This presentation yielded substantial interest in replicating the model in other academic institutions to address public health issues. Currently, faculty partners with colleagues to train on PBT implementation. The faculty continues to collaborate with the Agency as part of the PBT course and academic-community partnership. With this successful relationship, the faculty can improve the delivery of PBT coursework and use success stories to attract other PHAs for collaboration, refine PBT course design and delivery, secure federal and local funding, and measure impact of these projects on communities.

While the partnership between the faculty, student, and the Agency had substantial success during the PBT course and after with the continued academic-community partnership, as with any collaboration, partners had to navigate and overcome challenges. Some challenges that emerged throughout the follow-up period included timeline and resource constraints, barriers to implementing research into practice, and difficulties scaling the partnership. One key issue was the faculty, students’, and the Agency’s need to balance multiple priorities. For each stakeholder, the academic-community collaboration was only one facet of their professional or academic careers and required intentional investment and prioritization to support outcomes and with no earmarked time or financial support for the collaboration. The faculty and students operate on academic schedules, which often have breaks and periods of transitions built in, while the Agency often needed to maintain momentum on community research and initiatives. These mismatched timelines could contribute to gaps in progress. Due to the relatively short nature of graduate training (often 2 years or less) student turnover and career transitions also risk impacting the project’s timeline. This places an increased burden on the faculty and students who engage with the Agency as long-term partners to preserve institutional knowledge of the partnership and onboard and train new students to support its continuation. Similarly, with the recognition of the success of the academic-community partnership the faculty and the Agency increasingly had to balance managing their regular workloads with increased demands from inquiries from other state agencies. This required the faculty and the Agency to manage stakeholder expectations for these new initiatives and partnerships that arose from the success of PBT.

The primary goal of the academic community partnership was to support the Agency in translating planning from the course and research from the formative evaluation findings into practice. However, due to limitations within the Agency such as staffing, training, stakeholder buy-in, and operations, the findings required the Agency to assess its internal capacity. The faculty and students supported the Agency during the transition period as they collaborated on program development efforts. This included facilitating focus groups, analyzing community surveys, and identifying sustainable funding sources while adjusting the final deliverables from the course to align with practical considerations and constraints. With the support of the faculty and students, the Agency was able to build capacity, obtain additional funding, and subsequently implement new interventions.

There are some limitations of this case study. First, there are issues of generalizability. This was conducted on one small PHA with minimal hierarchical layers and results and beneficial outcomes may not be generalizable across all academic-community collaborations. In addition, this collaboration offers insights into the context of one large SPH within the specific framework of one MPH program’s curriculum design. This potentially limits the case study’s ability to capture the broader versatility of PBT, which extends beyond disciplinary confines and can be effectively employed across diverse programs and disciplines. While this case study presents some data on the long-term impacts of PBT on stakeholders, resource limitations prevented formal follow-up of outcomes and the inability to evaluate the sustained outcomes of the pedagogy for faculty, the Agency, and the students. In informal conversations, however, the Agency has reported using the deliverables for further program refinement and implementation and the students have reported the utility of the course and collaboration to success in the workforce.

In the ten years this course has been offered as a PBT course, however, and across a variety of PHAs, it is rare that the PHA, students, and faculty do not experience benefits of the collaboration beyond the 14-week semester. Second, the teaching team has been engaged in PBT for a long time and, over that time, has worked through some barriers and challenges to implementing PBT in the classroom. This experience, however, should not deter educators from considering PBT for their courses. The pedagogy is forever evolving with a foundation on real-time delivery, each semester bringing new collaboration and challenges. Finally, the case study represents a collaboration where the academic institution and PHA are in close geographic proximity, potentially supporting a successful and ongoing partnership. Over the almost three years of collaboration, all work except for one site visit has been done virtually highlighting the power of academic-community engagement in a virtual landscape.

Results of a PBT course vary depending on the topics addressed during the semester, the experience and background of the teaching team, the priorities of the PHA and the context of its community, and the level of investment of time and resources of the faculty. Each collaboration presents lessons learned that inform the establishment of future partnerships, course delivery, and sustainability of work post-semester. Some lessons learned from the establishment of the collaboration include defining clear goals and expectations for the community and its stakeholders, thoroughly reviewing the problem statement to ensure it meets course competencies, and streamlining engagement and communication processes.

During the course, touchpoints should be scheduled either in person or virtually to access PHA feedback toward tailored deliverables. Students should also be given multiple opportunities for feedback on their work to encourage a strong revise and resubmit process. Additionally, weekly email updates should be included across all stakeholders to keep them informed of the fluid nature of the course and expectations for upcoming classes. As illustrated by this case, sustainability of the work post-semester is possible and important. To ensure this, paid or credit-bearing opportunities should be created for students to continue working with the PHA. Post-semester meetings with the faculty and PHA should also be planned to assess needed resources and support to continue the work and how the faculty and SPPH can assist. Finally, student and PHA outcomes should be evaluated to redevelop and redesign the course for subsequent semesters. To add to this growing field of research, future examination of PBT should explore benefits and challenges in other disciplines using the pedagogy as well as longer-term outcomes for the PHA, faculty, alumni in the workforce, and the school, not just in graduate programs but undergraduate programs as well. Resources should be allocated to allow for longitudinal follow-up of outcomes for all stakeholders and to assess post-course outcomes including job readiness, intervention success for PHAs, and continued collaborations across partners. Strategic investment in such research will support increases in funding and capacity to redesign courses to be delivered with PBT and provide ongoing evaluation of the utility in practice.

As students graduate from public health programs and enter the workforce, there are continued concerns related to the undertraining of public health workers [43] . Assessments of current workforce needs indicate that skill gaps plague public health in resource management; forming and sustaining effective partnerships; implementing diversity, equity, and inclusion practices; and effectively communicating with internal and external stakeholders [42] [46] . These deficits in technical workplace skills align with core competency expectations that the Council on Education for Public Health (CEPH) requires for the accreditation of public health programs [8] - [11] . SPPH and faculty must support pedagogies that center these competencies and evaluate curriculum implementation to ensure the highest return on investment for students, faculty, and the field overall. Academic-community partnerships and PBT require investment from all stakeholders. However, the field demands that students and community partners enter the classroom to best prepare the next generation of the workforce. This investment is necessary to ensure the current and future workforce can respond to timely issues using innovation, collaboration, and immediate application.