Tag Archive for: What is evaluation?

Seven Signs Your Project Needs an External Evaluator

/in , /by

Seven signs your project would benefit from external evaluation

 

You don’t always need an external evaluator. Sometimes a clear internal framework, a few well-designed tools and protected time for reflection are enough.

But there are points when evaluation becomes difficult to hold alongside delivery. The project may have grown more complex, several partners may need a shared view, or the findings may now carry real weight with funders, boards and future decision-makers.

At that point, external evaluation is not simply about producing a report. It can bring structure to a complicated programme, protect the quality of the evidence and take some of the analytical burden away from a busy team.

Here are seven signs that outside support could be useful.

1. Your funder, board or partners need independent evidence

Internal monitoring can tell you a great deal, and delivery teams often hold vital knowledge about what is changing. But sometimes the people using the findings need a degree of independence too.

That may be because a funder expects an external view, a board will use the evidence to make a significant decision, or several partners need confidence that the findings do not reflect one organisation’s perspective alone.

An external evaluator can examine positive, negative and unexpected evidence, make limitations visible and distinguish what the project contributed from changes it cannot reasonably claim. Independence does not mean standing at a distance from the work. It means being close enough to understand it while retaining enough separation to test the story fairly.

2. The project has several strands, audiences or partners

Complex projects rarely arrive in one tidy shape. There may be different activities, places, audiences, delivery models and funding requirements. Partners may describe success differently or collect information in ways that do not easily fit together.

When that happens, the first value of evaluation is often clarity. An external evaluator can help the partnership agree the important questions, identify how the different strands connect and create a shared framework without flattening meaningful differences.

This is particularly useful in place-based, cultural, heritage, nature and community programmes, where outcomes develop through relationships and context as well as individual activities. The evaluator holds the overall picture while helping each partner understand their role within it.

3. You are collecting plenty of information but still cannot see the answer

Many organisations do not have a data shortage. They have attendance figures, spreadsheets, survey responses, case studies, meeting notes, photographs, analytics and informal feedback. What they lack is the time and structure to turn all of that into a clear judgement.

More collection will not necessarily solve that problem.

An evaluator can work backwards from what you genuinely need to know, assess what the existing evidence can answer and identify the important gaps. They can compare sources, look for patterns and differences, and test whether an appealing explanation is actually supported.

That process reduces noise. It helps teams move from ‘we have a lot of information’ to ‘we understand what this is telling us and what we need to do next.’

4. Evaluation keeps slipping behind delivery

This is extremely common. Delivery deadlines are immediate. Evaluation tasks feel as though they could happen next week, so next week becomes next month and important evidence is missed.

It is not usually because the team does not care. It is because the same people are managing activities, participants, partners, finances, communications and funder requirements. Evaluation becomes one more responsibility competing for attention.

External support creates ownership and rhythm. A good evaluator agrees responsibilities, introduces manageable tools, checks what is coming in and raises gaps while there is still time to respond. They do not expect delivery staff to become researchers. They create a process that fits around the work and keep hold of the parts that need specialist time.

This can bring a real sense of calm. Somebody is maintaining the framework, watching the evidence and keeping the final questions in view, rather than leaving the whole evaluation in an unfinished loop in the team’s heads.

5. You need to understand change, not simply count activity

Outputs matter. You need to know how many people participated, what was delivered and whether the intended audiences were reached. But those figures do not tell you, on their own, what difference the work made.

If the important questions involve confidence, wellbeing, belonging, skills, behaviour, organisational practice, environmental action or community change, you need evidence that can explore outcomes as well as activity.

An evaluator can help establish realistic baselines or comparison points, choose appropriate qualitative and quantitative methods, and consider change across different groups. They can also investigate how and why outcomes developed, including the conditions that helped or hindered them.

That prevents the final evaluation becoming either a list of impressive numbers or a collection of positive stories. Both can be valuable, but neither is enough on its own.

6. The findings will shape future funding or important decisions

Evaluation becomes especially valuable when the organisation needs to decide what to continue, change, scale or stop.

Perhaps a pilot is approaching its next funding stage. A successful model could be expanded. A partnership needs to agree its future direction. Or the organisation wants to make a stronger case for investment without overstating what the current evidence proves.

An external evaluator can help identify which elements are doing the most useful work, where outcomes are strongest, what remains uncertain and what would need to be in place for the approach to work elsewhere.

The aim is not to make every project look successful. It is to give decision-makers a credible account of what has been learned. Honest findings are often more reassuring than a report in which everything appears to have gone perfectly, because they show that the organisation is paying attention and knows how to improve.

7. The final evaluation needs to withstand scrutiny

A clear, attractive report is important, but reassurance comes from what sits underneath it.

Can each conclusion be traced to evidence? Have different sources been compared? Are sample limitations explained? Have differences between groups been examined where they matter? Are recommendations proportionate to what the findings can genuinely support? Has someone challenged the emerging interpretation before it becomes the final story?

External evaluation can provide analytical discipline and quality assurance. It also helps translate a complicated body of evidence into conclusions that funders, boards, partners and delivery teams can all understand.

The simplicity clients see at the end should be the result of difficult thinking being handled well—not difficult questions being avoided.

What does an external evaluator actually take on?

The answer depends on the project. External support could include refining a theory of change, developing an evaluation framework and practical tools, training delivery teams, undertaking selected interviews or fieldwork, checking data quality, analysing existing evidence, facilitating learning sessions and producing an independent report.

The evaluator does not necessarily need to collect everything. Often the strongest arrangement is a partnership: the delivery team gathers routine information at natural points, while the evaluator focuses on design, deeper enquiry, analysis, challenge and interpretation.

A good evaluator will help you decide what is proportionate rather than adding methods for the sake of making the evaluation look substantial.

You may not need a finished brief to get started with evaluation. Organisations sometimes wait to approach an evaluator because they think they need to specify the entire methodology first. You do not.

It is useful to know what the project is trying to change, who needs the findings, what evidence already exists and when important decisions or reports are due. The evaluator can then help shape the questions and recommend a realistic route.

At The Evaluator, we specialise in holding complexity without passing it back to the client. We create robust evaluation underneath and a process that feels straightforward on the surface. Our role is to bring clarity, honest independent judgement and evidence that people can confidently use.

If several of these signs feel familiar, send us your project outline or an early version of your brief. We can help you work out whether external evaluation would add value—and what level of support would be genuinely useful.

Often we can design a detailed evaluation quote in a 20 minute scoping call, and we are happy to do those. Why not reach out on info@theevaluator.co.uk ?

 

What Is Evaluation? A Practical Guide for Funders and Delivery Teams

/in /by

Evaluation Explained

What Is Evaluation and Do You Need It?

Evaluation turns delivery into decision-grade evidence. It helps you understand what changed, who benefited, why it worked (or didn’t), and what to do next.

If you’re delivering public-facing work, seeking funding, or managing multiple partners, evaluation is how you stay credible and in control.

What Evaluation Does

  • Measures: converts activity into insight so you can see what’s working and where to improve.
  • Proves impact: backs up your story with evidence funders can trust.
  • Strengthens funding: increases confidence that investment will lead to real outcomes.

What Funders Look For

Funders need to justify every pound. What gives them confidence:

  • Evidence, not anecdotes: stories land better when backed by data.
  • A clear Theory of Change: a believable pathway from activities to outcomes.
  • A learning mindset: reflection, adaptation, and improvement.
  • Transparency: honesty about challenges builds trust.
  • Alignment: wellbeing, heritage, environment, inclusion and other strategic outcomes.
  • Value for money: impact delivered efficiently and sustainably.
  • Proportionate methods: rigorous enough to be credible, realistic enough to deliver.

How Evaluation Builds Credibility

  • Focus effort where it matters: evidence shows what delivers results.
  • Prove and improve: demonstrate outcomes and strengthen delivery over time.
  • Stand out in bids: funders back organisations that can show learning and results.

When You Should Invest

For smaller projects, a light in-house approach can be sufficient — surveys, reflections and basic outcome tracking.

Once a project reaches roughly £50k–£100k+, independent evaluation often represents better value: more stakeholders, greater scrutiny, and more upside from credible evidence.

Need Practical, Funder-Ready Evaluation?

We help organisations measure what matters and turn findings into better decisions — without overcomplicating delivery.

Evaluation. Measurement. Impact.

What are the Indices of Multiple Deprivation?

/in , /by

What are the Indices of Multiple Deprivation?

If you have ever read one of our reports, you’ll likely see reference to something called the Indices of Multiple Deprivation, or ‘IMD’.

We use the IMD as a powerful tool to analyse and understand the barriers and socio-economic status of audiences, participants, and volunteers. We use postcodes to measure the IMD status, and this means that people find it easy to answer. This does mean that we are looking at the household status too.
The results give our clients an idea of which people they are attracting to their projects and what barriers they may be facing in life.

Statistically, people who live within areas of greater deprivation have more barriers in their lives and are less likely to engage with the arts and nature and are more likely to have greater issues with their wellbeing. That’s one of the reasons funders do like to understand if projects and organisations are reaching people from the most deprived areas.
How are the indices of multiple deprivation worked out?
The IMD is worked out using a range of factors which are assessed across the UK and given a score. The factors are listed below:

  • Income deprivation: how many people in the area are experiencing low-income levels.
  • Employment deprivation: what is the rate of unemployment in the area?
  • Education deprivation: what are the average qualifications within the area, and how easy is it to access quality education?
  • Health deprivation and disability: what is the average life expectancy and disability prevalence in the area?
  • Crime: how much crime occurs within the area
  • Housing deprivation: how affordable is housing within the area, and are there other barriers to people accessing housing
  • Living environment: what is the air quality like in the area, and are there green spaces?

These scores are compiled to give a ranking, which are then categorised into ten deciles. The entirety of England falls equally into one of ten deciles, with decile 1 indicating the most deprivation, and area 10 having the least deprivation.
If you were targeting deprived areas, you might want to reach 50% of people living in IMD1 and 2 for example. Or if you wanted a perfect section of the UK represented in your project, you might want to see 10% of your audience from each decile.

We think the IMD is a useful tool and aim to use it in every evaluation where relevant.

Learning about data; What is quantitative data?

/in , /by

What is Quantitative Data?

Quantitative data is data that can be counted or measured in numerical values.  As with qualitative data, there’s a good chance that you already have some collected for your organisation.

You might have collected some of the following:

  • Sign-in sheets
  • Feedback forms
  • Surveys
  • Polls
  • Social media statistics
  • Reports

We often find clients already have quite a bit of data they didn’t know they had collected!

The differences between primary and secondary quantitative data.

There is a distinction between primary and secondary quantitative data.  Primary data is the data that your organisation has collected directly, such as footfall counts or feedback forms. Secondary data is data someone else has collected, for example a national age profile, or a partner shares their footfall data. It can helpful for you to draw comparisons between your collected data and national averages to see how your organisation compares.

Overcoming the challenges of working with quantitative data

There are some challenges to working with quantitative data.  Often the biggest challenge is that it’s not collected in a format that makes it easy to compare to other collected data, or to secondary sources. The best solution for this is to plan in advance and use standardised questions at every opportunity to collect data. The answer format is also important so choosing a standard answer format will make it easier to compare data.

Tracking codes can be useful in identifying if you know the same person will be answering a survey multiple times so you can monitor their progress.  A tracking code can be created within a survey using data such as: a combination of a person’s date of birth and their initials.

It’s always important to date the data particularly for paper copies of surveys which makes identifying the event possible and the data relatable to that event. If one of your feedback forms reveals a problem with the venue or experience, you need to know the date on which that particular event happened to make sure you can address the problem. Don’t forget, feedback forms may be input or analysed as a batch of forms after a few months of collection so it may prove difficult to find out which venue the problem occurred at if you don’t have a way to check.

Managing personal data from surveys

GDPR (General Data Protection Regulation) are regulations which relate to how we retain and use personal data.  Within these regulations it is important to:

It’s important to maintain confidentiality and anonymity with personal information.  Recording date of birth and full name poses a risk to personal identity, however, recording only a date of birth is not identifiable. There are also additional regulations regarding collection data from under 16-year-olds. It is possible to collect identifiable information, but if you do so you need to ensure that the data is obtained with consent, is properly secured and then destroyed once no longer needed.

Thinking about when to collect data

Recording information in the moment is valuable so it can help to set up processes to ensure you don’t miss out! One tactic that works is to have a standard question you ask at the end of every event.  This, and the size of audience questioned can be collected for contextual purposes to see if the responses were representative of the larger audience.

Top tip – you don’t need to collect data from everyone!

Deciding how much data to collect

It’s important to consider whether sample sizes are large enough to provide you with sufficient data to base a decision on.  10% to 20% of the audience is usually a reliable sample size to base a decision on. If you hold many smaller events, it would be advisable to collect evidence from each event and consider it accumulatively to make decisions.