Teaching quality: does it come in bulk?

Ministers are concerned that despite large differences in the quality of novels, they all seem to cost the same. From now on, under the National Assessment of Fiction Framework (NAFF), the quality of novels will be scientifically measured according to the number of pages they contain, and prices will be set accordingly. Ministers are said to be delighted to have finally proved that Riders, Jilly Cooper’s 900-page epic, is three times better than Jane Austen’s insubstantial Pride and Prejudice.

This scenario makes as much sense as proposals to measure the quality of degree programmes by the number of contact hours they provide. Yet this is what is currently being explored as part of the teaching excellence framework (TEF) in England, the latest in a growing number of national higher education teaching excellence initiatives being developed around the world.

Producing a high-quality degree programme involves designing a range of experiences that allow students to develop an understanding of the knowledge they are engaging with, just as writing a novel involves designing a narrative that draws readers in and invites them to develop meaning from what they are reading. Clearly a certain number of hours are necessary for a course to be considered a degree, just as a certain number of pages are necessary for a story to be considered a novel. However, adding hours to a well-designed course will not improve it, any more than adding pages will improve a good novel. This point is supported by the 2016 UK Engagement Survey.

The other striking parallel is that both of these purported measures of quality are incredibly easy to rig. In the above scenario, publishers would surely reduce page sizes, increase font sizes or include longer blurbs or biographies. Similarly, faced with a measure of contact hours, universities would redefine what counts as one. Suddenly the definition would include academics’ office hours, regardless of whether anyone turned up. And what about throwing in that staff-student social event? There was definitely some informal feedback provided there. And while the regulators would react by attempting to impose their own definitions, scope for finessing them would always remain.

So why do contact hours seem to have such a hold on the Westminster government’s view of teaching quality? Why is it one of the few metrics it has instructed the Higher Education Funding Council for England to look into developing, while other more obvious measures, such as the expertise of teachers, have not even been mentioned? Partly it is because this approach appeals to a common sense notion of what students should be entitled to. It is easy to imagine the ministerial dinner party conversation: “Do you know, my Sophie went to Wessex and only got eight hours a week? Appalling – we must do something!” However, it is one thing to insist on a university applicant’s right to know how many contact hours they will receive on particular programmes. It is quite another to propose that figure as a valid measure of the quality of those programmes.

There are strong parallels here with the problems higher education researchers have had in proving the intuitively compelling relation between the amount of time students study and the quality of their learning. No relation could be found until the focus shifted to students’ perceptions of whether their workload was appropriate to support them in developing understanding. So while number of hours does not per se correlate with quality of learning, students’ perceptions of appropriate workload is one of its strongest predictors. This means it would be far better to examine whether students perceive they have the right number of contact hours to develop an understanding of their subject matter than to compare contact hours on different courses.

And the clear wider lesson for teaching excellence initiatives globally is that they need to be informed by research evidence, rather than by apparently convincing but deeply flawed common sense notions of the nature of excellent teaching.

This blog was also published in Times Higher Education.