DNP Program Evaluation Project: Kirkpatrick Model and Logic Model Applied to Nursing Education

Kirkpatrick Model: The Four Levels of Programme Evaluation

The Kirkpatrick Model evaluates educational and training programmes across four increasingly rigorous levels of evidence:

Level 1, Reaction: Did participants find the programme valuable, satisfying, and relevant? Measured by participant satisfaction surveys administered immediately after the programme. Reaction data are the easiest to collect and the least meaningful for programme improvement, a high satisfaction score does not guarantee that learning occurred or that practice changed. Most DNP programme evaluation projects that stop at Level 1 receive committee feedback that the evaluation lacks sufficient depth. Level 1 data are necessary but never sufficient as the sole evaluation outcome.

Level 2, Learning: Did participants gain the intended knowledge, skills, or attitudes? Measured by pre-post knowledge assessments, competency checklists, skills demonstration observation, or validated attitude instruments administered before and after the programme. Level 2 data demonstrate that the programme produced learning, a necessary precondition for behaviour change and outcome improvement. For nurse residency evaluation, Level 2 data might include pre-post scores on the Casey-Fink Graduate Nurse Experience Survey or a clinical knowledge assessment specific to the residency content areas.

Level 3, Behaviour: Did participants apply what they learned in their clinical practice? Measured 30 to 90 days after programme completion, using direct observation, chart audit, peer or supervisor report, or validated practice behaviour surveys. Level 3 data are the most difficult to collect in a DNP capstone because the timeframe required (30–90 days post-programme) may extend beyond the capstone implementation window. When Level 3 data cannot be collected within the project timeline, this is acknowledged as a limitation and a recommendation for follow-up evaluation is included in the Discussion chapter.

Level 4 (Results: Did the programme produce measurable organisational outcomes) improved patient safety metrics, reduced staff turnover, decreased adverse events, improved HCAHPS scores? Level 4 data are the most meaningful and the most difficult to attribute to a single programme. For a nurse residency programme, Level 4 data might include 12-month voluntary turnover rates for residency graduates compared to historical turnover rates, or unit-level patient safety event rates before and after residency implementation. Level 4 data for a single DNP capstone project are typically reported as trending data or retrospective comparisons, with appropriate acknowledgement of attribution limitations.

Logic Model: An Alternative Framework for Programme Evaluation

The Logic Model is a visual framework that maps programme inputs, activities, outputs, and outcomes in a left-to-right chain. For DNP programme evaluation projects, the Logic Model clarifies what the programme is supposed to produce at each stage and provides the structure for identifying what data to collect at each level. Logic Model components for a nurse residency programme evaluation: Inputs (funding, faculty time, simulation centre, preceptor network) → Activities (orientation modules, clinical rotations, simulation scenarios, reflective journaling) → Outputs (number of residents completing each module, preceptor contact hours, simulation scenarios completed) → Short-Term Outcomes (knowledge gain, competency assessment scores (Kirkpatrick Level 2) → Medium-Term Outcomes (practice behaviour change) Kirkpatrick Level 3) → Long-Term Outcomes (retention rate, patient safety metrics, Kirkpatrick Level 4). The Logic Model is typically presented as a table or figure in Chapter 3 (Methodology) and used to organise the data collection plan.

Programme Evaluation Project Examples by Track

Nurse Executive / Education: Nurse residency programme evaluation (Kirkpatrick all 4 levels; Casey-Fink Survey for Level 2; 90-day retention rate for Level 4). Preceptor development programme evaluation (preceptor confidence survey for Level 2; preceptee satisfaction for Level 3). Simulation-based education programme evaluation (knowledge pre-post test for Level 2; skills competency checklist for Level 2/3).

PMHNP / Behavioural Health: Staff trauma-informed care training evaluation (TICQ knowledge assessment for Level 2; direct observation of patient interactions for Level 3; patient safety event rate for Level 4). Suicide prevention training evaluation (Zero Suicide curriculum, pre-post knowledge test, C-SSRS documentation compliance for Level 3).

Population Health: Community diabetes education programme evaluation (diabetes knowledge questionnaire for Level 2; HbA1c at 90 days for Level 3/4). Community health worker training evaluation (competency checklist for Level 2; patient health literacy scores for Level 3).

IRB Considerations for Programme Evaluation Projects

Programme evaluation IRB pathways depend on the data sources. Using only existing programme records (pre-existing scores, attendance logs, retention data in HR system) without new data collection from identifiable individuals: QI non-research determination or exempt review under 45 CFR 46.104(d)(4). Administering new surveys to staff or participants as part of the evaluation: exempt review under 45 CFR 46.104(d)(2) if responses cannot be linked to identifiable individuals, or expedited review if identifiers are retained. Accessing patient records to extract Level 4 outcome data: typically exempt under 45 CFR 46.104(d)(4) if data are de-identified before analysis. Determine the IRB pathway before designing the data collection instruments, the pathway affects what data you can collect, not just how you document the collection.