Tag Archive for: data driven decisions

DATA DRIVEN DECISIONS

/in , /by

June is a busy month for our team as it is when we make all the data-driven decisions for the National Portfolio Organisations (NPO) funded by the Arts Council.

We offer an affordable service to help organisations collect all the data they need during the year for the Arts Council. We also help them import it into the funder’s portal, which opens between April and June. Whilst we are then  assembling all of the data, ready to fill in the portal, we analyse and report on all the data for the organisations, to enable them to make data-driven decisions. This is when we bring the data to life. Our data-driven decisions take the form of a team presentation followed by a report. We share trends shown over time, outputs, audience information and artist voices. We anonymise and share any staff issues and also provide staff and board demographic profiles. We look at ideas and spaces where organisations could shout about the great work they are doing (or do more of) alongside ways to improve. Some include practical operational tips to make sure organisations are using their limited resources and limited staffing in the best possible way, i.e. the most efficient way. The beauty of these presentations is that they are data-led and we don’t know what we will find until we analyse the information.

What is an NPO? The Arts Council explains “National Portfolio Organisations are leaders in their areas, with a collective responsibility to protect and develop our national arts and cultural ecology. Public investment brings public accountability, for us and for the organisations we invest in, and this is reflected in our expectations.”

Organisations do have to be accountable in terms of detailed data. It can be quite tricky to understand the requirements too, for example, in 2024/2025 there have been a lot of changes, with the launch of a new system for one part of data collection, called ‘Illuminate.’

The Evaluator has been offering this service for a number of years, and our NPO cohort has grown significantly since 2023. We love working with arts organisations to make sure we figure it out for them. The Arts Council is keen for organisations to use their own data more, so it’s a win-win relationship.

Are you  an NPO organisation who has recently been through a challenging time in finding data, assembling the right information and putting all your data into the portal when it opened? If so then why not give us a call? We might be able to help you and it is much better value than you might expect. In fact, our packages have a standard cost of £2,400 plus VAT each year. You can call us on 01756 532 538 or email info@theevaluator.co.uk

If you are interested in this idea but are not quite sure you want to talk yet, then why not join our mailing list?

The Evaluator Mailing List

The Evaluator Mailing List, for updates and newsletters.

Please wait...

<span style="color: #339966;">Thank you for signing up. If you'd like to get in touch directly please email info@theevaluator.co.uk</span>

We only mail out about 3 newsletters per year, so you definitely won’t feel bombarded, but it is a great way to keep in touch and learn about our approach to evaluation and data in general.

The Evaluator Values 

/in , /by

At The Evaluator, we have always had a clear idea of what we do and why we do it. In 2024, we have clearly defined that with a statement of values.

The values represent who we are at heart, and what we do for our clients, as well as underpinning all of our work.

What are business values?

Business values are described as the core principles and standards that guide a company’s actions, decisions, and behaviours. They represent what a company stands for and are essential in shaping its culture, brand identity, and strategic direction. These values influence how a company interacts with its employees, clients, funders, and the community in general.

By clearly defining and consistently using values, organisations can create a strong, positive culture and a sustainable competitive advantage.

The Evaluator Business Values

 

Image of the evaluator ruler branding and the words creatively simple honest and useful

Our values are shown on the image above and are; Honest, Useful and Creatively Simple.

Core Value for The Evaluator: Honest

Honesty matters to us. When decisions are to be made about data – people need to feel that the results are honest. We need to be honest about how we have collected data – what questions did we ask, and where did we ask them. We are by nature, honest people, who say the truth, and that is the secret to our success. Honesty builds trust, and our clients know we will tell the truth. That means funders can trust our impartial evaluation too, and that has many benefits for our clients when it comes to applying for funding in the future.

Core Value for The Evaluator: Useful

What we are really passionate about is getting our clients to use data to make decisions! We want people to be inspired and enthusiastic about data, and what it can do for organisations. That means, we spend a lot of time focussing on what data people need and how they can use it. We train clients and the organisations we work with on how to collect the correct data, and how to understand the data. We often collect data for people too. What is essential to this value (usefulness), is that we collect the right amount of data which we can use. We do not collect large amounts of data that is stored and never looked at. We collect the right amount of data, which is often less than people expect, and use what we have collected. The secret to powerful data is about how it is analysed and used, not how much of it there is!

Core Value for The Evaluator: Creatively Simple

Data can be complex. We spend a lot of time and energy making it creatively simple for people to work with us. Examples of creatively simple ways to collect data are things like – sticker boards, marbles, hula hoops and giant dice. These have all featured in our work! There can also be creatively simple ways to demonstrate complex results and personal experiences to clients. This is a value that we are always working on in order to improve.

Why did we choose these values?

Founder and Director, Kirsty Rose Parker, explains “We have been working hard behind the scenes on articulating what it is that makes us stand out. Sharing our core values is the first step of this. We have looked carefully at all the steps that make up an evaluation with us. From listening to people at the beginning, to being creative with our methods at evaluation design time. We regularly challenge ourselves on how to be simpler, how to be more creative and also on how our work can be honest and useful. These values are now an everyday part of The Evaluator – helping us to grow as a team, whilst retaining our quality service and products.”

Why do values matter in evaluation?

These are our values, but we also often include client’s values in the evaluations we plan. We have helped clients decide what values work for new projects in the past, and then used those values to form the materials we create for clients. Having a short and memorable set of values is a great way to give new teams a frame of reference for their work. We look forward to a value-driven future creating great partnerships and measuring even more values.

 

If you are interested in measuring your values or helping to define them, then please get in touch on info@theevaluator.co.uk or call 01756 532 538 to book an informal discussion.

If you like what you have read here, but are not yet ready for a discussion, you can keep in touch with us by joining our mailing list or connecting with Kirsty on Linked In.

The link for our mailing list can be found here.

The link for connecting with Kirsty on Linked In can be found here.

What are the Indices of Multiple Deprivation?

/in , /by

What are the Indices of Multiple Deprivation?

If you have ever read one of our reports, you’ll likely see reference to something called the Indices of Multiple Deprivation, or ‘IMD’.

We use the IMD as a powerful tool to analyse and understand the barriers and socio-economic status of audiences, participants, and volunteers. We use postcodes to measure the IMD status, and this means that people find it easy to answer. This does mean that we are looking at the household status too.
The results give our clients an idea of which people they are attracting to their projects and what barriers they may be facing in life.

Statistically, people who live within areas of greater deprivation have more barriers in their lives and are less likely to engage with the arts and nature and are more likely to have greater issues with their wellbeing. That’s one of the reasons funders do like to understand if projects and organisations are reaching people from the most deprived areas.
How are the indices of multiple deprivation worked out?
The IMD is worked out using a range of factors which are assessed across the UK and given a score. The factors are listed below:

  • Income deprivation: how many people in the area are experiencing low-income levels.
  • Employment deprivation: what is the rate of unemployment in the area?
  • Education deprivation: what are the average qualifications within the area, and how easy is it to access quality education?
  • Health deprivation and disability: what is the average life expectancy and disability prevalence in the area?
  • Crime: how much crime occurs within the area
  • Housing deprivation: how affordable is housing within the area, and are there other barriers to people accessing housing
  • Living environment: what is the air quality like in the area, and are there green spaces?

These scores are compiled to give a ranking, which are then categorised into ten deciles. The entirety of England falls equally into one of ten deciles, with decile 1 indicating the most deprivation, and area 10 having the least deprivation.
If you were targeting deprived areas, you might want to reach 50% of people living in IMD1 and 2 for example. Or if you wanted a perfect section of the UK represented in your project, you might want to see 10% of your audience from each decile.

We think the IMD is a useful tool and aim to use it in every evaluation where relevant.

Learning about data; What is quantitative data?

/in , /by

What is Quantitative Data?

Quantitative data is data that can be counted or measured in numerical values.  As with qualitative data, there’s a good chance that you already have some collected for your organisation.

You might have collected some of the following:

  • Sign-in sheets
  • Feedback forms
  • Surveys
  • Polls
  • Social media statistics
  • Reports

We often find clients already have quite a bit of data they didn’t know they had collected!

The differences between primary and secondary quantitative data.

There is a distinction between primary and secondary quantitative data.  Primary data is the data that your organisation has collected directly, such as footfall counts or feedback forms. Secondary data is data someone else has collected, for example a national age profile, or a partner shares their footfall data. It can helpful for you to draw comparisons between your collected data and national averages to see how your organisation compares.

Overcoming the challenges of working with quantitative data

There are some challenges to working with quantitative data.  Often the biggest challenge is that it’s not collected in a format that makes it easy to compare to other collected data, or to secondary sources. The best solution for this is to plan in advance and use standardised questions at every opportunity to collect data. The answer format is also important so choosing a standard answer format will make it easier to compare data.

Tracking codes can be useful in identifying if you know the same person will be answering a survey multiple times so you can monitor their progress.  A tracking code can be created within a survey using data such as: a combination of a person’s date of birth and their initials.

It’s always important to date the data particularly for paper copies of surveys which makes identifying the event possible and the data relatable to that event. If one of your feedback forms reveals a problem with the venue or experience, you need to know the date on which that particular event happened to make sure you can address the problem. Don’t forget, feedback forms may be input or analysed as a batch of forms after a few months of collection so it may prove difficult to find out which venue the problem occurred at if you don’t have a way to check.

Managing personal data from surveys

GDPR (General Data Protection Regulation) are regulations which relate to how we retain and use personal data.  Within these regulations it is important to:

It’s important to maintain confidentiality and anonymity with personal information.  Recording date of birth and full name poses a risk to personal identity, however, recording only a date of birth is not identifiable. There are also additional regulations regarding collection data from under 16-year-olds. It is possible to collect identifiable information, but if you do so you need to ensure that the data is obtained with consent, is properly secured and then destroyed once no longer needed.

Thinking about when to collect data

Recording information in the moment is valuable so it can help to set up processes to ensure you don’t miss out! One tactic that works is to have a standard question you ask at the end of every event.  This, and the size of audience questioned can be collected for contextual purposes to see if the responses were representative of the larger audience.

Top tip – you don’t need to collect data from everyone!

Deciding how much data to collect

It’s important to consider whether sample sizes are large enough to provide you with sufficient data to base a decision on.  10% to 20% of the audience is usually a reliable sample size to base a decision on. If you hold many smaller events, it would be advisable to collect evidence from each event and consider it accumulatively to make decisions.

 

 

 

 

 

 

 

 

New Client: Love Withington Baths

/in , /by

The Evaluator is delighted to be working with a new client, Love Withington Baths.

The Evaluator is working with the community leisure centre to carry out a Social Return On Investment (SROI). You can read our blog post all about SROI here, but it’s basically a formal process of identifying what matters to people taking part, starting with the changes they identify. We are looking forward to chatting away to people using the centre, and figuring out what changes it has led to in people’s lives. It’s a really well used centre, so we should get to talk to lots of people!

You can see more about Withington Baths here. 

Learning about data; Working with qualitative data

/in /by

From time to time, we share a director’s blog post, where we share some learning about how to use your data (information) better. This is a post all about how to use qualitative data, that’s the information that is made up of words rather than numbers.

The chances are that your organisation already has a bank of this data but it may not be well organised and easily accessible. Below is a list of some of the many ways you might have collected qualitative data, possibly without realising it.

How credible is your qualitative data?

There is a distinction between primary and secondary data and it’s important to understand what you have collected as the credibility differs between the two.  Primary data is more credible and robust.  It is the data that people told you or have written themselves.  Secondary data is that which is overheard in discussion or something that someone has heard and told you about.

To check the credibility of your secondary data, you can apply the Rule of 3. If you’ve heard it from 3 different places or if it’s come from 3 different people independently then it can be considered as credible and representative of a thought or opinion.

How to make the qualitative data more usable? 

We use word or phrase frequency analysis to evaluate our qualitative data to look for patterns of frequency of words to identify common themes.  This can be done online using free tools, like this one,  and they just count the words for you.  We often use the top 10 words or phrases.  You can also use the ratio of positive to negative words that are used or how far down the feedback the first negative word appears.  Using these methods helps to quantify data and make it more digestible and can be used in marketing or to track changes over time.

Mind maps can be useful to illustrate and develop on the themes identified. You can just draw these freehand to have a look at what the main themes are.

Word clouds provide a visual representation at a glance of the qualitative data and this is also a resource that can be sourced freely online.  The most common words appear the biggest in the cloud, making this data easier for the reader to understand visually.

Top tip – try to be objective, it can be hard to hear negative comments but it is how we improve and know what to fix, and they’re often in the minority compared to positive comments.

How to use and share qualitative data?

This data can be used in many different ways:

We also use qualitative data to create case studies which illustrate people’s journeys and direct engagement.  Case studies can be shared at board level and to show case your project’s work for marketing purposes.  Case studies appeal to a wide audience and are particularly useful in attracting funders as voices are recorded and reflected in these studies.

In our experience as evaluators, a case study from a project that we evaluated got shared with funders (Green Recovery) who sent it on to The National Lottery Heritage Fund who then sent it on to the Department of Work and Pensions.  Case studies are impactful in that they record real voices and can attract publicity and raise awareness of change.

 

If all of this sounds like too much work and you don’t have time, get in touch. We are happy to have a chat about your individual requirements and to see how we could help.