Tag Archive for: Impact Measurement

What Happens After You Appoint an External Evaluator?

What happens after you appoint The Evaluator?

Appointing an external evaluator can feel like handing an important part of your project to people you do not yet know very well. You may be wondering how much time the process will take, what your team will be expected to do, whether the evaluator will understand the work quickly enough, and whether problems will be spotted before the final report.

Those are reasonable concerns. A strong proposal matters, but clients also need confidence in what happens after the contract is signed.

Our job is to make evaluation feel held: clear responsibilities, sensible tools, regular contact and enough challenge behind the scenes to produce findings that stand up to scrutiny. This is the process you can expect from us.

1. We begin by getting the important things out of people’s heads

The first stage is inception. We review the brief, funding requirements, project plans, existing monitoring and any previous research. Then we talk through what the project is really trying to achieve, what is already happening and where the uncertainties sit.

This is often where different strands begin to join up. Projects can involve several partners, audiences, activities, outcomes and reporting requirements. We create a shared picture, agree priorities and make decisions about what the evaluation can realistically answer.

This is often the point when people start to feel excited about the evaluation: the project becomes clearer, the possibilities begin to emerge, and the team can see how the process will support their work. We also begin agreeing the practical arrangements and responsibilities, building these out as the evaluation develops.

2. We turn ambition into answerable questions

Funding applications often contain broad and overlapping ambitions. Our role is to translate those into a focused set of evaluation questions and, where useful, a theory of change.

That means identifying the route from activity to outcomes, the assumptions underneath it and the evidence that will help us judge progress. We also review your targets and deliverables, checking that everyone understands what needs to be counted, how it will be recorded and who will be responsible. Where possible, we establish baselines or sensible comparison points and make clear where evidence will be indicative rather than conclusive.

This work creates the spine of the evaluation. Without it, projects can collect a great deal of information and still struggle to demonstrate what they delivered or answer the questions that matter.

3. We build tools that work in the real world

Next, we develop the evaluation framework and practical toolkit. This could include monitoring templates, surveys, interview guides, observation tools, participant reflection, demographic questions or creative in-person methods.

The right tool depends on the setting. A short community activity, an outdoor festival, a long-term volunteer programme and a strategic partnership cannot all be evaluated in the same way.

We design first for the people who will actually use and respond to the tools, as well as for the evidence the evaluation needs. If you are working with people experiencing deprivation, people facing language barriers, children or people with complex disabilities, we will adapt both the method and the way it is delivered so that participation is practical, appropriate and meaningful.

That might mean using simpler language, visual or creative approaches, supported conversations, shorter questions or accessible alternatives to a conventional survey. We aim to create methods that work in real settings—not polished, bureaucratic tools that look impressive on paper but place unnecessary demands on participants or delivery teams.

4. We make responsibilities manageable

External evaluation works best as a clear partnership. Your team may hold attendance records, capture routine feedback or introduce us to participants. We may lead deeper interviews, focus groups, observation, analysis and independent reporting.

We agree that division early and provide support where needed. This might include a short training session, written guidance, worked examples or a regular check-in. The aim is not to turn delivery staff into researchers. It is to make the information they gather consistent and useful without adding unnecessary weight to their jobs.

5. We check the evidence while the project is live

One of the biggest risks in evaluation is waiting until the end to discover that an important question was not asked or that data has been recorded inconsistently.

We build in proportionate progress and data-quality checks. We look at what is coming in, whether different groups are being heard, where response rates are weak and whether delivery changes affect the original framework. If something is not working, we raise it while there is still time to adapt.

These conversations are not about catching clients out. They are about protecting the final evaluation and helping the project learn during delivery.

6. We analyse, compare and challenge

Analysis is more than summarising a spreadsheet or selecting good quotations. We look for patterns, differences between groups, changes over time and explanations for what we are seeing. We compare monitoring data, surveys, interviews, observation and other sources rather than treating each in isolation.

We also look for evidence that complicates the emerging story. Where samples are limited, baselines are missing or other factors may have contributed to change, we say so. Conclusions are tested within the wider team before they become part of the report.

That internal challenge is a core part of our quality assurance. Accessible writing is the final layer; underneath it sits careful analysis and professional judgement.

7. We turn findings into something people can use

The report should give funders and boards confidence, but it should also help the people delivering the work. We structure findings around the questions and outcomes that matter, show how conclusions were reached, and make limitations clear without allowing them to obscure useful learning.

Recommendations are grounded in the evidence and proportionate to the organisation’s capacity. Depending on the commission, we may also present findings, run a learning session or produce concise visual material for wider audiences.

Our aim is not to leave a large document behind and disappear. It is to help the right people understand what the evidence means and what to do next.

What happens when projects change?

Projects change because funding is delayed, staff move, activities evolve, partners join or audiences respond differently from expected. Evaluation has to be structured without becoming rigid.

We keep the central questions in view, record agreed changes and adapt tools or timing where that protects the usefulness of the work. If a change affects scope or budget, we explain that early rather than allowing a surprise to appear at the end.

What about capacity and continuity?

Clients appoint The Evaluator as a consultancy, not an isolated individual. Each project has clear leadership and project management, with wider team involvement in delivery, analysis, review and cover. Our systems hold actions, deadlines and decisions so that continuity does not rely on one person remembering everything.

That team structure matters. It gives clients consistent contact while bringing more than one perspective to important findings. It also means quality and progress can be protected through busy periods, leave and the ordinary realities of working life.

What we need from you

Good evaluation does require collaboration. We will need timely access to relevant documents and data, honest conversations about delivery, help reaching participants or stakeholders, and enough notice of key events and changes.

In return, we take responsibility for holding the evaluation: keeping the framework coherent, making sense of evidence, identifying risks, asking the difficult questions and turning complexity into clear, useful findings.

If you are planning an evaluation and want to understand what the process would look like for your project, send us your brief—or talk to us before the brief is finished. We will help you work out a proportionate route from the questions you have now to evidence you can confidently use.

Illuminate changes FOR NPO’s


Illuminate Changes for NPOs

Early in 2025, Arts Council England announced some significant changes to their reporting system, Illuminate.

To paraphrase – they have announced that although Illuminate has been widely adopted, with over 500,000 survey responses collected since its launch, there is no longer the same requirement to meet audience targets for data collection in detail.

All NPOs and IPSOs must continue using Illuminate to report on funded activity and audience or participant numbers every quarter, as this data contributes to official statistics. While organisations are still encouraged to gather insights on their audiences, they will not be required to use Illuminate audience surveys in 2025/26.

What This Means for Organisations

You can now choose how to collect data and determine what works best for you. This also means you can ask things in a different order, reword questions, and make your data collection truly work for your organisation.

You might be wondering what to do now. Well, we would always recommend collecting audience data – and then using that data to make decisions. In fact, if you are unsure, we’d say collect less information but use it more. The worst outcome for everyone is collecting lots of information and then not doing anything with it.

Below are some pros and cons of collecting audience data to help you decide. And remember – we’re more than happy to help with this. Our full National Portfolio Organisation (NPO) package, which provides annual support to collect data, complete the annual return, manage quarterly uploads to Illuminate, and then analyze and present that information back to you, starts at just £2,400 (+VAT) per annum. It’s very good value for money, as we do this for a number of organisations and it is surprisingly affordable.

Pros of Collecting Audience Data

✔️ Powerful Insights: It’s a great way to understand who is engaging with your work, what they enjoy, and who might be missing from your reach.

✔️ Informed Decisions: Gathering this insight helps you pinpoint where to improve and where to focus your energy for the greatest impact.

✔️ Relevance & Engagement: It ensures your work remains relevant, accessible, and engaging to the widest possible audiences.

✔️ Audience Connection: People appreciate the opportunity to share their thoughts and feel they have a voice in shaping the future of your organisation.

✔️ Building Inclusivity: Acting on audience feedback strengthens relationships and helps build a more inclusive, successful organisation.

Cons of Collecting Audience Data

⚠️ Time Commitment: Collecting data takes time – for staff to gather it, for teams to analyze it, and for audiences to complete surveys.

⚠️ Survey Fatigue: People are often asked for feedback in many areas of life, so they’re more likely to respond positively if they see their input genuinely valued and acted upon.

⚠️ Meaningful Use: Data should shape decisions and improve experiences rather than becoming a tick-box exercise. When purposeful, data collection feels less like a task and more like a tool for growth and connection.

Our mantra is simple: collect less, but use it more. There’s nothing worse than gathering huge amounts of data only to discover there’s no capacity to make practical use of it. Focusing on meaningful, targeted information means every piece of data serves a purpose and drives real impact.

You’re not alone in this our NPO data return package includes dedicated analysis time, so you can make the most of the insights you collect without feeling overwhelmed. We’re here to help you turn data into action.

If you have any questions or want more information, contact us at info@theevaluator.co.uk or call 01756 532 538 and we’ll be delighted to see how we can help.