What really happens in an education research trial?

Two EEF-funded trials of NCETM programmes are now underway, focusing on Mastering Number at Reception and KS1, and the Secondary Non-specialist Teachers SKTM Programme. The NCETM’s Director for Research, Evaluation and Impact, Dr Vivien Townsend, shares how the trials are progressing, the challenges encountered and why independent evaluation matters for pupils and schools.

Earlier this year, Dr Jen Shearman explained in a blog post, Going for gold standard, why the NCETM and the Maths Hubs Network were committed to two EEF-funded evaluations  in 2025/26. 

It’s crucial that schools put themselves forward to take part in external evaluations. They enable rigorous, independent assessment of programmes designed to improve pupils’ education. The schools involved in the Randomised Controlled Trials (RCTs) of two Maths Hubs programmes this year are providing objective evidence of the impact of our work. This is vital to understand if programmes are effective, so we can learn how to improve what we do and ensure resources to improve maths education are well spent. 

Months of planning, and hundreds of schools and teachers working with their Maths Hub, have ensured that the trials were able to start this academic year. A heartfelt thank you to all the schools involved.

A reminder of what we signed up to

The two programmes being evaluated in 2025/26 are: Mastering Number at Reception and KS1, with a particular focus on the impact on pupils in the Reception year; and the Secondary Non-specialist Teachers SKTM Programme, with a focus on the impact on Year 8 students taught by participating teachers.

Why we signed up

We want to understand how the activity of the NCETM and the Maths Hubs Network supports the mathematical development of all pupils, especially those from disadvantaged backgrounds. These independent evaluations will also provide schools with additional impact data when choosing to participate in the future.

Randomised Controlled Trial (RCT) methodology

A Randomised Controlled Trial (RCT) is the gold standard for any evaluation project, and this is the approach the EEF adopts for its funded evaluations. RCTs offer a combination of fairness, control and reduced bias, which means the final results are more reliable, and can be used with greater confidence in decisions about future practice or policy. RCTs are quite different from the NCETM’s usual evaluation methodology, which is based on case studies, interviews and participant surveys with teachers and schools. Both approaches have their benefits and drawbacks. 

An RCT compares schools in an intervention group (who are allocated a place on  the programme in 2025/26) with those in a control group (who do not receive a place on the programme in 2025/26 but are guaranteed a place the following year, subject to continued DfE funding). 

To participate, schools submitted a Memorandum of Understanding (MoU) and then completed some evaluation activities (including submitting pupil data and completing teacher surveys) which would establish a baseline. In October, the schools in both evaluations were randomised, so that half of the schools recruited by each hub (or cohort) were placed into the intervention group, and half into the control group.

The RCT methodology presents three challenges:

• the existence of a control group can negatively impact recruitment
• maintaining good relationships can be difficult, and
• the additional administration and logistical demands of being in a trial places a burden on the NCETM and Maths Hub teams.

We believed that, by the NCETM and the Maths Hubs Network working together, these challenges could be mitigated. In this blog post, I’ll consider each in turn and share the reality.

Challenge 1: no-one wants to be in the control group

The numbers recruited to both programmes in the hubs involved in an EEF-funded evaluation is lower than in the hubs not involved, and hubs have reported that the RCT – and the possibility of not getting the programme in 2025/26 – was a factor. We know that some schools chose to wait a year rather than take the risk of being disappointed after randomisation!  

Schools in the control group are the real stars of the RCTs. We know that this outcome is not what they hoped for, but their ongoing participation in evaluation activities is essential if the evaluation is going to show the impact of the programmes. Thank you so much to these schools – we really couldn’t do this without you. Your willingness to participate and work with us is building an important evidence base for future Maths Hubs’ work. We are deeply grateful.

Challenge 2: maintaining relationships

Saying ‘no’ to schools has been difficult for Maths Hub teams, whose fundamental purpose and drive is to provide support  to schools in their region. As well as disappointing schools in the control group, hub colleagues also had to say ‘no’ to some schools earlier in the process, because despite a school being keen to access a programme, they were ineligible for the trial.

A particular challenge has arisen where the evaluation was the first time that a school had engaged with the hub, and then the school was randomised into the control group. Because of the RCT, hubs are limited in terms of what they can offer to soften the blow of missing out, and the NCETM Communications Team is supporting with suggestions.

Challenge 3: increased administration and logistical demands

Hub teams have been incredibly supportive of the evaluations, but this has undoubtedly put increased pressure on operations teams; to say they have stepped up is an understatement. Colleagues have distributed school information sheets, become experts in explaining RCTs, collected paperwork, chased missing data, and generally corralled, reassured and sympathised with schools in their region. We could not have asked for a better response from the network.

I’m also incredibly grateful to my NCETM colleagues – especially those in the Operations, Communications and System Leadership teams – who have rallied around to support the evaluations. We’ve also been lucky to work with evaluators, NFER and Sheffield Hallam University, who have supported our collective understanding. Any questions about the evaluations can be directed to them at masteringnumber@nfer.ac.uk or sktm@shu.ac.uk.

We are underway

Despite a delayed start to the programmes (to accommodate randomisation), the schools in the intervention group are now participating in the programmes as ‘normal’. As always, local leaders of mathematics education (LLMEs) are doing a superb job of leading the programmes and supporting individual participants to achieve the intended outcomes.

The schools in the control group are urged to continue with their ‘business as usual’. At the end of the year, the control schools participating in the Mastering Number evaluation will be asked how they have supported children in Reception and KS1, and control schools in the Secondary Non-specialist Teachers SKTM evaluation will be asked about the support provided to the participating teacher. In both cases, this is expected to be what the school has done historically.

What will happen next?

At the end of the year, all schools in the evaluations will be asked to complete teacher surveys and submit pupil data. These will be compared to the baseline data gathered at the start of the year to identify changes. This contributes to the Impact Evaluation (IE). 

Alongside the RCTs, an Implementation Process Evaluation (IPE) is taking place across the year. This is designed to understand how the programmes are experienced by participants, and to capture how this translates into classroom practices. To explore this, evaluators have designed measures of fidelity to the programmes (e.g., attendance at training, use of materials) and are observing some national and local sessions. Some schools will also be invited to be the subject of a case study  (which may involve school-based observations and interviews with teachers). The relationship between fidelity to a programme and impact on pupils is one we’re particularly interested to understand.

We will celebrate good news (of course!) but we will also welcome any findings that help the NCETM and the Maths Hubs Network to become even better at what we do.

And finally, the external evaluations have already had an impact on my team, which has been renamed Research, Evaluation and Impact in recognition of the NCETM’s commitment to research. It’s been a challenge, but I’m proud that, through these EEF-funded evaluations, the NCETM is actively contributing to the growing evidence base of educational interventions.