Tag Archive for: inclusive evaluation

What Happens After You Appoint an External Evaluator?

What happens after you appoint The Evaluator?

Appointing an external evaluator can feel like handing an important part of your project to people you do not yet know very well. You may be wondering how much time the process will take, what your team will be expected to do, whether the evaluator will understand the work quickly enough, and whether problems will be spotted before the final report.

Those are reasonable concerns. A strong proposal matters, but clients also need confidence in what happens after the contract is signed.

Our job is to make evaluation feel held: clear responsibilities, sensible tools, regular contact and enough challenge behind the scenes to produce findings that stand up to scrutiny. This is the process you can expect from us.

1. We begin by getting the important things out of people’s heads

The first stage is inception. We review the brief, funding requirements, project plans, existing monitoring and any previous research. Then we talk through what the project is really trying to achieve, what is already happening and where the uncertainties sit.

This is often where different strands begin to join up. Projects can involve several partners, audiences, activities, outcomes and reporting requirements. We create a shared picture, agree priorities and make decisions about what the evaluation can realistically answer.

This is often the point when people start to feel excited about the evaluation: the project becomes clearer, the possibilities begin to emerge, and the team can see how the process will support their work. We also begin agreeing the practical arrangements and responsibilities, building these out as the evaluation develops.

2. We turn ambition into answerable questions

Funding applications often contain broad and overlapping ambitions. Our role is to translate those into a focused set of evaluation questions and, where useful, a theory of change.

That means identifying the route from activity to outcomes, the assumptions underneath it and the evidence that will help us judge progress. We also review your targets and deliverables, checking that everyone understands what needs to be counted, how it will be recorded and who will be responsible. Where possible, we establish baselines or sensible comparison points and make clear where evidence will be indicative rather than conclusive.

This work creates the spine of the evaluation. Without it, projects can collect a great deal of information and still struggle to demonstrate what they delivered or answer the questions that matter.

3. We build tools that work in the real world

Next, we develop the evaluation framework and practical toolkit. This could include monitoring templates, surveys, interview guides, observation tools, participant reflection, demographic questions or creative in-person methods.

The right tool depends on the setting. A short community activity, an outdoor festival, a long-term volunteer programme and a strategic partnership cannot all be evaluated in the same way.

We design first for the people who will actually use and respond to the tools, as well as for the evidence the evaluation needs. If you are working with people experiencing deprivation, people facing language barriers, children or people with complex disabilities, we will adapt both the method and the way it is delivered so that participation is practical, appropriate and meaningful.

That might mean using simpler language, visual or creative approaches, supported conversations, shorter questions or accessible alternatives to a conventional survey. We aim to create methods that work in real settings—not polished, bureaucratic tools that look impressive on paper but place unnecessary demands on participants or delivery teams.

4. We make responsibilities manageable

External evaluation works best as a clear partnership. Your team may hold attendance records, capture routine feedback or introduce us to participants. We may lead deeper interviews, focus groups, observation, analysis and independent reporting.

We agree that division early and provide support where needed. This might include a short training session, written guidance, worked examples or a regular check-in. The aim is not to turn delivery staff into researchers. It is to make the information they gather consistent and useful without adding unnecessary weight to their jobs.

5. We check the evidence while the project is live

One of the biggest risks in evaluation is waiting until the end to discover that an important question was not asked or that data has been recorded inconsistently.

We build in proportionate progress and data-quality checks. We look at what is coming in, whether different groups are being heard, where response rates are weak and whether delivery changes affect the original framework. If something is not working, we raise it while there is still time to adapt.

These conversations are not about catching clients out. They are about protecting the final evaluation and helping the project learn during delivery.

6. We analyse, compare and challenge

Analysis is more than summarising a spreadsheet or selecting good quotations. We look for patterns, differences between groups, changes over time and explanations for what we are seeing. We compare monitoring data, surveys, interviews, observation and other sources rather than treating each in isolation.

We also look for evidence that complicates the emerging story. Where samples are limited, baselines are missing or other factors may have contributed to change, we say so. Conclusions are tested within the wider team before they become part of the report.

That internal challenge is a core part of our quality assurance. Accessible writing is the final layer; underneath it sits careful analysis and professional judgement.

7. We turn findings into something people can use

The report should give funders and boards confidence, but it should also help the people delivering the work. We structure findings around the questions and outcomes that matter, show how conclusions were reached, and make limitations clear without allowing them to obscure useful learning.

Recommendations are grounded in the evidence and proportionate to the organisation’s capacity. Depending on the commission, we may also present findings, run a learning session or produce concise visual material for wider audiences.

Our aim is not to leave a large document behind and disappear. It is to help the right people understand what the evidence means and what to do next.

What happens when projects change?

Projects change because funding is delayed, staff move, activities evolve, partners join or audiences respond differently from expected. Evaluation has to be structured without becoming rigid.

We keep the central questions in view, record agreed changes and adapt tools or timing where that protects the usefulness of the work. If a change affects scope or budget, we explain that early rather than allowing a surprise to appear at the end.

What about capacity and continuity?

Clients appoint The Evaluator as a consultancy, not an isolated individual. Each project has clear leadership and project management, with wider team involvement in delivery, analysis, review and cover. Our systems hold actions, deadlines and decisions so that continuity does not rely on one person remembering everything.

That team structure matters. It gives clients consistent contact while bringing more than one perspective to important findings. It also means quality and progress can be protected through busy periods, leave and the ordinary realities of working life.

What we need from you

Good evaluation does require collaboration. We will need timely access to relevant documents and data, honest conversations about delivery, help reaching participants or stakeholders, and enough notice of key events and changes.

In return, we take responsibility for holding the evaluation: keeping the framework coherent, making sense of evidence, identifying risks, asking the difficult questions and turning complexity into clear, useful findings.

If you are planning an evaluation and want to understand what the process would look like for your project, send us your brief—or talk to us before the brief is finished. We will help you work out a proportionate route from the questions you have now to evidence you can confidently use.

Meet Victoria

MEET VICTORIA

Meet Victoria – The People-Focused Evaluator Making Stories Count

At The Evaluator, we believe that great evaluation isn’t just about numbers and data—it’s about people. That’s where Victoria Huntriss comes in. With a background as a qualified counsellor and clinical supervisor, Victoria brings something truly special to our evaluation projects: the ability to really listen. Her experience in community work, social care, and mental health support means she knows how to create safe spaces where people feel comfortable to share their experiences.

Whether she’s conducting interviews, gathering case studies, or interpreting qualitative data, Victoria makes sure that every voice is heard and valued.

Her skills in person-centred and cognitive behavioural therapy make her a natural at understanding the impact that projects have on individuals. She has worked in substance misuse programs, bereavement support, trauma counselling, and more—so she knows that every story is unique.

Victoria is especially passionate about advocating for neurodivergent and autistic individuals, ensuring that evaluation is inclusive, accessible, and meaningful.

As part of The Evaluator team, she takes a people-first approach, turning real-life experiences into powerful insights that help to shape better, more impactful projects. With her on board, we’re not just collecting data—we’re telling the human story behind it.

Inclusion at The Evaluator: Figuring it out for everyone

Accessibility is something that is really important to our organisation. We want to figure things out for people and projects, and part of that is figuring out how to do that for the widest range of people possible!

We are a disability led organisation which is one reason we are passionate about being inclusive. We try hard to follow the social model of disability where we understand that we need to adapt the world around a person’s disabilities and access needs rather than expect them to adapt. We recognise that disability is more than wheelchair access, it’s a wide range of adaptations, many of which are easy and free. We try and include as wide a range of access needs as we can think of in each project and are always willing to adapt to make sure our processes and materials suit the client and participant needs.

We are still learning and imagine we will be for all time, we know we don’t have all the answers but here are some of the ways we try to be more inclusive, all of which are actions we have carried out.

  • When we run 1:1 interviews we can offer to send questions in advance, plan in comfort breaks and run these at times that suit people – including evenings and weekends
  • We can use large font sizes on emails or text clients or schedule regular zoom calls
  • We use easy to read fonts on our surveys and our infographics and use contrasting colours in graphs as much as possible
  • We are happy to communicate however people prefer – whether that is by email, by text, by zoom, by phone call – or even mixed methods, for example mixing text and video call by using zoom captioning
  • We can run sessions when people have supporters or interpreters available and make our timetable flexible
  • We provide large print versions of our final reports and executive summaries on request
  • We have worked hard to ensure all our visuals are compatible with screen readers, use alt-text as much as possible and break reports into sections using headings and subheadings for clarity
  • We can provide surveys in alternative ways – for example using Google Forms or as paper copies
  • We worked with a client to use sign language videos on our surveys to make sure people understand the questions asked and adapted our questions to suit British Sign Language (BSL) rather than expecting BSL to adapt to our questions
  • We offer adaptations to our interview processes when hiring and offer flexible working conditions and hours

We have three current and recent clients who have taught us a lot about being more accessible.

  1. Attitude is everything

Attitude is everything have been passionate about including people with disabilities for a long time now. They have a few cultural approaches which we love and which we have tried to follow. Their overarching tag line is ‘nothing for us, without us’ and therefore we have learned to test materials and not make assumptions without asking. It helps us keep flexible. The other way AisE work is to help people make changes from where they are now, not to criticise what they have or haven’t done. Again, that’s something we use now with all our clients, and try to embody ourselves – helping them to make positive progress from wherever they currently are, using data driven decisions of course.

We particularly love their ‘how we think about disability’ approach, which you can read here. 

  1. Deaf Rave

Troi Lee from Deaf Rave was a great client to work with, and we adapted our communication styles, using zoom on mornings when a sign-language interpreter was available. We learned about the Deaf Culture, and attended an event in person to communicate with sign language interpreters therefore adapting our style. Troi described working with Kirsty Rose Parker from The Evaluator as “if you need a evaluator report done.  She the one!  I have not seen any better report than anyone else has done.”

Troi is a passionate advocate for Deaf music and regularly works with the BBC, most recently taking part in the BBC music introducing festival to put on London x Deaf Rave. You can see more about that festival here.

  1. VocalEyes

VocalEyes are a charity who work with people with many different disabilities including visual impairments and people who use screen readers. This has meant making a number of changes to our project evaluation including software changes, allowing for access needs such as emailing questions and using zoom captioning. This has allowed us to learn more about inclusion and practical methods to adapt our styles.

To provide an evaluation report which can be understood by the widest possible range of people, we are currently learning how to make the best accessible report we can and are working on developing spoken video versions of our executive summaries in final reports. Please do ask us if you would want a version of this!

VocalEyes have just produced ten ways to attract deaf, disabled and neurodivergent visitors to heritage sites which is well worth a read. Click here to look at those ten tips.