FFT impact report

Mathematics Mastery evaluation: GCSE attainment 2018-19

Report from FFT Education Datalab, June 2021

Read the report in full

What did the FFT report look at?

The report evaluated the impact of signing up to the Mathematics Mastery secondary programme on GCSE mathematics grade, in two outcome years: 2018 and 2019.

What was the methodology?

The evaluation used what is known as a quasi-experimental design. This involves comparing the outcomes of pupils who went to a school that took part in Mathematics Mastery to those of pupils from a matched comparison group of statistically similar schools. This approach mimics what would be done in a formal experiment such as a randomised control trial.

FFT selected schools that were similar with respect to:

  • Three years of historic GCSE attainment data in maths (where available)
  • Three years of historic GCSE overall attainment (e.g. Attainment 8, where available)
  • % pupils eligible for free school meals
  • % pupils with a first language other than English
  • Mean prior attainment at KS2 for each cohort on roll

Only mainstream state-funded schools in England were considered for the comparison group. We also excluded selective schools, as none of the treated schools were in this category. The comparison group was constructed using a technique known as covariate balancing propensity score matching (CBPS). This procedure reweights the non-participating schools such that, in aggregate, the set of non-participating schools and the set of participating schools balance (i.e. match) on the pre-treatment characteristics listed above.

Having calculated weights for each school, FFT estimated the effect of the programme using weighted pupil-level data. We used multilevel models, nesting pupils within schools and controlling for a number of confounding variables related to the outcome: gender,
whether their first language was English, whether they were eligible for free school meals

      What does the FFT report tell us?

      The FFT evaluation found:

      • conclusive evidence to show that Mathematics Mastery had a positive impact on GCSE mathematics grade in 2018/19
      • the estimated impact on pupils in the treated group of schools was the equivalent of around 1/8 of a grade, or one month of additional progress.

      Find out more about Mathematics Mastery's secondary programme