Program Evaluation

Program evaluation assesses, after a curriculum or educational program has been running, whether it actually achieved the outcomes it was designed for — using what data, compared against what original goals. It comes after the fact: Training Needs Analysis establishes the gap a program should exist to close, Curriculum Design builds the program to close it, and program evaluation is the step that checks, once the program has actually run, whether it did.

Key components

• Original goals — the specific outcomes the program was designed to produce, retrieved from the original design rather than reconstructed after the fact from whatever data happens to be available.
• Outcome data — evidence of what actually happened: assessment results, completion rates, follow-on performance, gathered in a form that can be compared against the original goals rather than just described in general terms.
• Comparison or baseline — some basis for judging whether the outcome data represents success, whether that's a pre-program baseline, a comparison group, or a stated target set in advance.
• Implementation fidelity check — whether the program was actually delivered as designed, since a program that wasn't implemented as planned can't fairly be judged against outcomes it was never really given the chance to produce.
• Findings and recommendation — a clear judgment (continue, revise, discontinue) with the reasoning that led to it, not just a data summary left for someone else to interpret.

Fidelity before outcome

A program evaluation that finds disappointing outcomes and stops there is missing the question that actually determines what to do next: was the program run as designed. A curriculum that looked strong on paper can produce weak outcomes for two very different reasons — either the design itself doesn't work, or the design was never actually delivered as intended (teachers skipped components, pacing collapsed under real constraints, the assessment used to measure outcomes wasn't the one the design assumed) — and these call for opposite responses. Weak outcomes from a faithfully implemented program are a signal to revise or discontinue the design; weak outcomes from a program that drifted from its own design during implementation are a signal to fix implementation before concluding anything about the design at all.

Checking fidelity means gathering some evidence of what was actually delivered — classroom observation, teacher self-report against the program's own components, materials actually used — not just assuming implementation matched the plan because the plan existed. Skipping this step is how a genuinely sound program gets discontinued for a delivery problem, or a genuinely broken one gets kept running because nobody checked whether it was ever properly tried.

Common pitfalls

• No original goals to compare against — evaluating a program against goals reconstructed after the fact, rather than what it was actually designed to achieve, makes almost any outcome look defensible.
• Outcome data with no baseline — reporting results with nothing to compare them to, so there's no way to tell whether the numbers represent success, failure, or business as usual.
• Skipping the fidelity check — judging a design by outcomes without first confirming the program was actually delivered as designed.
• Evaluation timed too early — measuring outcomes before the program has had enough time to produce them, especially for programs whose effects compound over multiple years.
• Findings with no recommendation — a data summary that describes what happened without stating what should happen next, leaving the actual decision to someone with less context than the evaluator had.

Learn more

• Curriculum Design for designing the program this evaluation assesses after the fact.
• Curriculum Mapping for auditing coverage and alignment across courses, a different kind of check than whether a program met its outcomes.
• Training Needs Analysis for confirming, before any program exists, what gap it needs to close — the goals a later evaluation will measure against.