Imagine a TEF that actually measured teaching excellence…

There has been much debate about the Teaching Excellence Framework since it was initially proposed. Some have called for an outright rejection, while others have come up with ideas about the principles it should follow. But I am interested in how to design an effective system to measure and improve the quality of learning and teaching across the higher education system. What would that actually look like?

There are four things that need to be understood:

1. We need to recognise that any form of measurement is expensive and so, to avoid wasting money, it needs to lead to improvements in practice. 
2. Crucially, we need to accept that no measure is precise. Metrics are sledgehammers rather than lasers and will, at best, offer broad indications of quality. Any high-precision instrument that we are offered will be lubricated with vast quantities of snake oil.
3. We need to take account of what we know from more than 40 years of research into learning and teaching in higher education, but also what we know about how institutions respond when they are assessed on performance measures.
4. We need to recognise that this involves designing an assessment system, rather than arguing over individual metrics.

Ingredients of a fit-for-purpose TEF

• The criteria will be based on a research-informed view of high-quality teaching. We need a clear basis on which to include and exclude each criterion. There are a number of potential frameworks that could be used for this, for example, the Teaching and Learning Research Programmes 10 principles of teaching and learning or the UK Professional Standards Framework for Higher Education. 
• The individual metrics informing the assessment system need to be part of a collective and coherent system. This involves relating the metrics to the overall criteria and showing how they measure something that is identified as a necessary element of high quality teaching. 
• The metrics should be measures of the quality of teaching offered, rather than the prestige of institutions or the profile of their students. To improve their scores, institutions would have to improve their teaching practices. 
• The overall criteria should remain stable but the actual metrics should change every year. They should be drawn from a very large public bank of potential metrics. This is important so that institutions focus on improving the quality of their practices rather than fixing individual metrics.
• The judgments offered by the TEF should be relatively simple. They should be focused on the subject level rather than the institutional level because institutions can, clearly, have programmes of variable quality.

Lack of vision

Judged by these criteria, the current Year 2 version of TEF performs OK. In particular, benchmarking the metrics by students’ entry characteristics and flagging statistically significant differences are improvements on university league tables. However, not focusing on the subject level is a major flaw in the Year 2 TEF’s ability to offer valid indications of quality (although this is pencilled in for Year 4).

But the impact of not basing the system on a coherent, research-informed vision of teaching quality becomes even more apparent in the discussion of future metrics.

Measures that we know do not relate to the quality of learning and teaching, such as contact hours or “teaching intensity”, are being pushed by the government. Measures that we know are crucial, such as the expertise of teachers or the ways students are transformed by knowledge, are not even part of the conversation.

The issue is that if the TEF is not based on a systematic view of high-quality teaching, then there is no basis on which to include or exclude potential measures of excellence. As a result, an opportunity to develop the quality of teaching across the sector – and give students more valid indicators of the quality of different courses – will be lost to ever-changing ministerial hunches about what good university teaching looks like.

This blog was also published in The Guardian.