Evaluation continued

One way to do this is for the evaluator and program people to develop a very good description of:

·      What the outcomes should be

·      How the program will get there

·      Why the program leads to the outcome.

This description helps to identify how the program should lead to the outcome, why the program activities should lead to the outcomes, and where to evaluate the program to check whether it does.

This method is called a program theory.

“A program theory explains how and why a program is supposed to work. ... It provides a logical and reasonable description of why the things you do – your program activities – should lead to the intended results or benefits.”

A useful tool to help work with the program theory is a logic model, which visually shows the program theory, how all the program goals, activities, and expected outcomes link together.

Use the program theory or logic model to come up with evaluation questions.

·      Does the program have a positive outcome?

·      Are people satisfied?

·      How could the program be improved?

·      How well is the program working?

·      Is the program working the way it was intended to work?

However, there are limits to program theory and logic models:

·      Models are linear, programs are complex, interactive

·      Models are static, programs may change over time.

·      Models may not take unexpected consequences into account

·      Models may not account for conflict, power, control issues

·      Theory or model assumes the model is correct.

Use program theory and logic models, but be flexible, and open to change and feedback. Review and revise them often, as necessary.

Introduction to Evaluation #1

 What is evaluation?

When beginning an evaluation, program people will often want the answer to this question:

·      Does the program work? And how can it be improved?

However, there are many equally important questions

·      Is the program worthwhile?

·      Are there alternatives that would be better?

·      Are there unintended consequences?

·      Are the program goals appropriate and useful?

Program evaluation is “…the systematic assessment of the operation and/or outcomes of a program or policy, compared to a set of explicit or implicit standards as a means of contributing to the improvement of the program or policy…”

The systematic assessment

An evaluation is a systematic assessment. Evaluations should follow a systematic and mutually agreed on plan. Plans will typically include the following:

·      Determining the goal of the evaluation: What is the evaluation question; what is the evaluation to find out?

·      How will the evaluation answer the question: What methods will be used?

·      Making the results useful, how will the results be reported so that they can be used by the organization to make improvements?

The first part of the evaluation is to determine the question.

“Assessment of the operation and/or outcomes of a program or policy”

Evaluations can generally answer two types of questions:

1. What is the outcome of the program? Did the program have any impact? Was there any improvement in people's lives?

2. How did the program get to that outcome? Did the program have some set of procedures? Were these procedures followed, were the procedures reasonable, was there a better way to get to the outcomes?