How to evaluate social programs?

Evaluating social programs is not just good practice but a responsible one!

When we create a social program or intervention that will impact people’s lives, we also have a responsibility to monitor and evaluate the actual impact these programs or interventions are having.

There are two main reasons why evaluating social programs is not just good practice but responsible practice:

Where your program has positive social impacts and is making a real difference in people’s lives, quality evaluation provides the evidence needed to demonstrate its value. With fewer and fewer resources available, this provides decision makers with much needed information and reduces the risk of the program being undermined or de-funded. If you have a program that you know is making a real difference, the best thing you can do for the people receiving its benefits is to evaluate its effectiveness and provide funders with the necessary evidence to keep supporting it.
There is often an assumption that programs are having a positive impact because the people designing and delivering them have good intentions. However, it is also possible that programs with good intentions can have negative consequences or they are not being as effective as possible.

The best way to understand the impacts of a program and improve its performance is to evaluate it.

Why people don’t evaluate programs?

There are many valid reasons why people don’t evaluate their programs. The main reasons I often hear is that:

there is a lack of capacity (time, money or resources)
organisations don’t have the capability (the skills and knowledge to undertake evaluation) to evaluate a program.

Viv Warren helping clients to evaluate their social programs

Here are the key steps in evaluating a program to help you get started, noting that they will not be able to fill all the gaps or contribute to capacity.

Key steps to evaluate your social program

Design your evaluation as early as possible and ideally when you are first developing your social program. This ensures that you are clear about what you are evaluating and can collect data throughout the program lifecycle. If you have been running a program for some time and have not yet started your evaluation, it may not be ideal, but it is never too late to start!

step 1 – Conduct a theory of change

The first step in designing an evaluation process is to understand and articulate the program logic and its theory of change. A Theory of Change outlines what you hope the program will achieve over time and the value you provide to your beneficiaries/clients, it is aspirational. View Theory of Change worksheet

step 2 – UnderSTAND stakeholder interest

A program will often have many changes that can be monitored and evaluated. However, it is really important to focus your efforts on the most important changes. Deciding what to focus on needs to consider the stakeholders that are interested in your program and answer their key questions. Assessing who they are and who are the most important will help to focus the evaluation. View stakeholder analysis worksheet

step 3 – Design your evaluation process

There are many methods that can be used for evaluating your program. The evaluation approach you design should be commensurate with the size and complexity of your program. It is really important to strike the right balance between the reliability and validity of the data collected and the amount of time and money required. For example, a very large program may warrant a survey with high statistical validity. However, a smaller sample size (with lower statistical validity) may be sufficient for a smaller program.

For information about evaluation methods, visit the Evaluation Toolbox or see The Magenta Book.

step 4 – Collect data

Once you have designed your evaluation approach, you need to prioritise the development of your data collection tools and ensure that you collect the information needed for your evaluation. It is really important that the data collection tools are designed and implemented to the highest professional standards to ensure that the data is robust, reliable, valid and credible. It is often at this stage that most attention is needed as time can easily slip-by delivering your program. Having an implementation plan for your evaluation that details the steps in your evaluation and the timeframe is helpful for ensuring you collect the necessary data.

step 5 – Report on evaluation outcomes

Consolidating and analysing the data collected and organising it into a report can seem a daunting exercise. The most important thing to remember is that your job at this stage is to transform ‘data’ into ‘knowledge’ and presenting the information in a way that answers your stakeholders questions as concisely as possible (preferably in one or two pages for key decision makers).

Consider using a simple and visual presentation format such as the Extreme Presentation Method.

step 6 – Reflect, learn and improve

Allocate some time at the end of the evaluation process to meet with the program stakeholders to discuss the outcomes of the evaluation and identify strengths and opportunities for improvement. This key step will help to inform the future direction of the program and improve its performance.

step 7 – Review evaluation approach

Following the evaluation, it is helpful to reflect on the evaluation process itself and identify what worked well and what didn’t work so well. The evaluation approach can then be adapted to changes in the direction of the program that emerged from the ‘reflect, learn and improve’ stage and learning from the evaluation process itself.

Although it is tempting to invest all the program’s resources into helping beneficiaries or clients, it is important to remember that evaluation is critical for understanding and providing evidence of the impacts of a program. If done well, evaluation may help to secure future funding and improve the program’s performance.

By Viv Warren, Viv Warren Consulting

Using social science to inform decision-making

If you need support in gaining stakeholder and community insights to inform decision-making, contact Viv Warren Consulting. Our social research delivers meaningful insights, fostering an understanding of stakeholder and community perspectives that drive informed and effective decision-making for organisations.

Partnering with organisations, we use a suite of qualitative techniques, such as in-depth interviews and focus group discussions, as well as quantitative tools like surveys and statistical analysis, ensuring a comprehensive understanding of stakeholder and community views.

Viv Warren Consulting Social Research services