Home / Overview of Ofsted’s approach to ‘summary evaluations’ of MATs.
Overview of Ofsted’s approach to ‘summary evaluations’ of MATs.

In December 2018, the Secretary of State wrote to Ofsted confirming the process for summary evaluations of multi-academy trusts. The summary evaluations are not inspections and are defined by Ofsted as being “when inspectors evaluate the quality of education provided by a MAT and leaders’ contributions to this, drawing on previous inspections of individual academies within that MAT.”

 

Stage One

The ‘previous inspections of individual academies’ referenced in Ofsted’s definition above are carried out through ‘batched inspections’, a process that has been in place since 2015, and is now referred to by Ofsted as ‘stage one’ of the summary evaluation process. Batched inspections – an overview of which can be accessed here – are where a number of schools across a trust are subject to individual inspections within a short or relatively close period of time. Until December 2018, these individual inspections all took place within one week of one another. The process has been changed in the following way:

  • The scheduling of school inspections that form part of the MAT summary evaluation over a period of up to two terms rather than over a week; (although it should be noted that Ofsted has retained an option to conduct all of these individual inspections over the period of a week should it wish to)

Ofsted will notify academies that are inspected during stage 1 in the usual way. The inspections will be carried out under section 5 or 8 of the Education Act 2005 and in accordance with the framework for school inspection, school inspection handbook and the school inspection handbook: section 8.

Ofsted will, however, be mindful that some powers and responsibilities may not lie with individual schools. In its guidance it states: “During each section 5 or section 8 inspection, inspectors will consider the same evidence and make the same judgements exactly as they would under any other section 5 or 8 inspection. In doing so, inspectors recognise that responsibility for some aspects of the education that the academy provides may be held at academy level and some may be set by the leaders of the MAT. Therefore, they will gather evidence accordingly.”

In preparing for the inspection of individual schools within the MAT, it is clear that HMI will review a range of information, not least schemes of delegation “and other information published by the MAT, such as its approach to curriculum design and implementation, recruitment and continuing professional development of its staff.” Other information may include previous inspection outcomes, complaints and regional intelligence. HMI will share this information with the inspectors leading the academy inspections so that the latter are clear about the way the MAT is structured and operates.

Ofsted will use this intelligence to help it pursue its lines of enquiry around the impact of the use, quality and impact of support and direction that the MAT gives to its academies.

Ofsted is clear that “If any of this evidence relates to the wider MAT rather than the standards and outcomes of the academy being inspected, inspectors will consider it as part of the evidence base for stage 2 (the MAT summary evaluation) but will not take it into account in reaching judgements about the academy.”

IMPLICATIONS FOR MATS: Ensure your websites are clear on how your trust works and operates, outlining key information in an accessible manner.

 

Will Ofsted engage with the central MAT team during stage one?

During a section 5 or 8 inspection, the guidance states that as well as having separate meetings with the academy principal and the chair (or equivalent) of the body responsible for governance of the academy, the inspectors will engage with “if available, a representative of the MAT (to be determined by the MAT)”

 

Progress to stage Two?

It is clear that not all MATs that are subject to ‘stage one’ will then proceed to stage 2, the point at which it goes beyond the individual inspection of a number of the trusts schools and engages with the central MAT in order to undertake a summary evaluation. Stage two is at the discretion of the Regional or National director, and in taking the decision as to whether to proceed to stage 2 they may have reference to:

  • MATs that have a high number of academies that are performing well;
  • MATs in which lower-performing academies have shown improvement;
  • MATs that have weak inspection outcomes.

Ofsted will not notify the MAT about the summary evaluation until it completes stage 1, so there is time to re-appraise the decision to carry out a review.

The lead HMI will notify the MAT about the summary evaluation (stage 2), normally up to five working days before the start. Subject to the availability of the MAT’s chief executive or equivalent, HMI should ask to speak to the most senior executive officer available.

The purpose of the notification call is for the lead HMI to:

  • inform the MAT of the summary evaluation and the telephone surveys with academies that were not inspected during stage 1
  • alert the MAT to the fact that there could be some on-site visits during the week, to be agreed with MAT and academy leaders
  • make arrangements for summary evaluation activities, including preparations for discussions with senior leaders, such as trustees, members of the trust board and directors, senior officers and other staff/partners
  • outline the process to be followed and provide an opportunity for the MAT to raise any initial questions
  • request that relevant evidence be made available during the on-site week, making clear that Ofsted does not expect information to be provided in any particular format and that the MAT does not need to do any activities specifically for the review
  • confirm the dates of the summary evaluation activity and remind the MAT that the resulting letter will be published on Ofsted’s website.

 

What does stage two of the ‘summary evaluation’ involve?

One the first day, HMI should meet briefly with the chief executive, their equivalent or delegate to introduce the inspection team and any other attendees, confirm arrangements for meeting with key staff, and to confirm arrangements for providing feedback during and at the end of the review and discuss practical issues

The following elements will usually be included in a MAT summary evaluation:

  • published inspection outcomes from the batched inspections carried out at the academy level and the evidence bases from those inspections
  • a telephone survey of academies that were not inspected during stage 1
  • a visit to the MAT that includes discussions with leaders
  • on-site visits to some of the academies within the MAT.

In its guidance to inspectors, Ofsted is clear that: “MAT structures are diverse, reflecting the values and ambition of each MAT. Ofsted has no fixed view on what constitutes the best arrangement. Inspectors should therefore avoid advocating any particular structure or arrangement. However, inspectors should feel confident commenting on the effectiveness of the arrangements that are in place, based on the evidence they gather. The Department for Education has published guidance about governance structures in MATs, which includes information about effective arrangements in relation to checks and balances.”

 

When carrying out summary evaluations, inspectors may take into account the following:

  • if available, a summary of any self-evaluation regarding the overall quality of education within the MAT academies and arrangements for supporting and enhancing this
  • any strategic plans for education within the MAT, including details of partnership arrangements, commissioning and brokerage and any evaluation reports
  • documentation about arrangements for monitoring, challenge, intervention and support of trust academies
  • any documentation or strategy relating to curriculum design and implementation
  • any documentation or strategy relating to pupil behaviour, including the use of exclusion and any forms of off-rolling
  • any documentation or strategy relating to the recruitment, training and continual professional development of academy staff at all levels
  • any data sets about performance of trust academies and any analyses thereof
  • case study material regarding targeted school improvement work and its impact on trust academies, including curriculum plans
  • a list of key staff and their roles and responsibilities
  • information regarding strategies used to improve the quality of teaching (or education) across the trust academies and evidence of their impact
  • information regarding strategies used to improve the standards of governance in individual academies and across the MAT.

 

There is no expectation that the MAT should prepare evidence specifically for the benefit of inspectors.

Discussions and the evidence presented may be wide ranging. However, they should be firmly focused on impact, both for the pupils served by the MAT and in terms of raising standards and the quality of education offered, including through improving the performance of its member academies.

The questions that follow are not intended to be a checklist or exhaustive. In their meeting with senior leaders, undertaking on-site visits, and through telephone calls. Inspectors will tailor their lines of enquiry to the circumstances of the MAT. HMI are likely to explore:

  • the impact the trust has on its academies, including what elements of education are uniform across the trust, where the trust allows academies more autonomy and how the trust reduces underperformance
  • how the trust monitors its academies, especially how it understands the quality of education being offered and its impact
  • how policies are made, implemented and reviewed, including how curriculums are designed and reviewed, and the impact they have
  • what the trust’s intentions and ambitions are, especially in relation to its curriculum, and how these are communicated, understood, implemented and monitored across the trust
  • how the trust identifies its strengths and weaknesses, including in terms of the teaching within individual academies, and how the trust responds
  • the way in which MAT leaders lead the trust, including how staff recruitment, development and management work across the trust
  • how the MAT trustees work with the MAT leadership, including how they set priorities and how they constructively challenge leaders.

Further details on the arrangements for on-site visits and telephone surveys (and the recording of evidence) can be accessed here: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/769153/Summary_evaluations_of_multi-academy_trusts_070119.pdf

It is important to note that on-site visits to the schools not inspected as part of stage 2 must be made with the agreement of the MAT leaders and schools.

 

Safeguarding dimension

When evaluating a MAT, inspectors may consider how far MAT leaders and managers have put in place effective arrangements to:

  • identify children and learners who may need early help or are at risk of neglect, abuse, grooming or exploitation
  • help prevent abuse by raising awareness among children and learners of safeguarding risks and how and where to get help and support if they need it
  • help those children who are at risk of abuse and need early help or statutory social care involvement, keeping accurate records, making timely referrals where necessary and working with other agencies to ensure that children and learners get the help and support they need
  • manage allegations about adults who may be a risk, and check the suitability of staff to work with children, learners and vulnerable adults

 

Feedback and summary evaluation letter

HMI carrying out summary evaluations will offer oral feedback on emerging findings to senior leaders, such as trustees, members of the trust board and directors, senior officers and other staff/partners to promote professional dialogue.

Before the end of the summary evaluation (stage 2), HMI should ensure that the MAT is clear:

  • about Ofsted’s view of the relative strengths and weaknesses of the quality of education and leadership and management offered across the MAT
  • that the main points provided orally in the feedback will be referred to in the text of the summary evaluation letter, subject to quality assurance
  • about the post-review procedures leading to the publication of the letter
  • about what to do if the MAT has a complaint.

After inspectors have reached their provisional findings, they will present and briefly explain these to senior executives and board representatives of the MAT.

Before publication, the HMI leading the summary evaluation (stage 2) will provide the MAT with the draft letter. The MAT will normally have five working days to make any comments (including factual accuracy comments), on the draft letter.

The lead inspector will set out the findings from the summary evaluation activity in a letter to the MAT chief executive, or equivalent, and copied to the SoS and the chair of the trust board. The letter will highlight specific areas of strength and areas where the support and challenge that the MAT offers its academies can be improved. It will report the range of evidence considered and outline the most recent inspection outcomes for all academies within the MAT. Summary evaluations do not result in graded judgements. The judgements that inspectors make about the MAT’s impact, strengths and weakness will always be in narrative form. The letter will be published on the Ofsted reports website. The format of the letter will be as follows:

  • introduction
  • summary of main findings – the headlines that are explained in more detail in the letter
  • range of evidence
  • context
  • main findings
  • findings on safeguarding
  • recommendations
  • Letters will set out previous inspection outcomes for all academies within the MAT.