Global BUzz

BIS published its Third Report of Session 2015-16, The Teaching Excellence Framework: Assessing quality in Higher Education today.  The report is an interesting read, highlighting the excellent international reputation of the UK HE sector as second only to the US, and that as such the Government needs to ensure that changes are introduced and implemented properly to avoid adverse consequences.

‘It is this global context within which HEIs operate and expectations about excellence are set. The UK has one of the best university systems in the world, but it is an increasingly competitive market and we need to ensure this good system is getting even better’, the report said.  Understanding the competitiveness of the sector, the report recognises a heightened Government responsibility ‘to establish new mechanisms to measure properly the level of teaching quality’.  It continued, ‘there is much at stake. A poorly designed or rushed TEF will not serve students, HEIs, Government or the taxpayer and could negatively affect reputations. Equally, a well designed and implemented TEF could provide a model for other nations and enhance the UK’s already strong position.’

The tone of the report emphasised general support for the underlying principles of TEF, but equally recognised ‘the legitimacy of concerns about the practical details of implementation, given the proposed pace of implementation.’

The timeline at present for introducing the TEF is as follows:

15 January 2016 Green Paper consultation closes

January to March 2016 TEF level 1 decisions made but not announced (status of QA reviews and fee uplift)

Spring 2016 Government response to Green Paper consultation

Spring 2016 TEF technical consultation

Spring 2016 TEF level 1 decisions announced

September 2016 Response to technical consultation

October 2016 Publish technical guidance for providers and panel members

October to December 2016 Providers apply

January to February 2017 Panels undertake assessments

March 2017 Moderation

April 2017 Results to providers and published, to inform decisions relating to 2018–19

***

The report covers quite a lot of ground, however the main issue is about how the TEF might legitimately signify quality, and whether the proposed link between TEF and tuition fees is justified at present.  There is a strong underlying perception that focusing on teaching quality will not only help meet student expectation but will help the UK maintain its leading position in global HE.

However, there is concern and doubt surrounding the lack of consensus and agreement of ‘definitions or recognised measures of teaching quality’.  The report states ‘the challenge is to identify those metrics which most reliably and accurately measure teaching quality, as opposed to other factors that contribute to the results achieved by students.’

The Government’s three proposed metrics – employment, retention and satisfaction – are looked at deeper and commented on:

‘Employment/destination:

• There is no evidence that graduate destinations or salary levels are good proxies for teaching quality.
• Women, BME students and those from disadvantaged backgrounds tend to earn less
• Privately educated students are more likely to get a “good degree” than state educated students or those from disadvantaged backgrounds and earn more three years after leaving university
• Graduate destination can be affected by the regional economy
• Graduate destinations are influenced by subject more than teaching quality
• Some jobs are not classified as ‘graduate’, despite requiring higher level qualifications.

Retention

• Retention rates are affected more by other factors relating to the university experience than by teaching quality (eg student intake)

Satisfaction

• The National Student Survey is currently being revised, with some changes to the questions expected to be introduced in 2017.
• Concerns about the limited use of the NSS included:
  • Satisfaction is not the same as teaching quality;
  • There is no indication of “why” a course is rated as it is or commentary on what matters to students;
  • Results tend to vary by discipline;
  • NSS is open to gaming and does not demonstrate quality’

The Committee also points to ‘associated risk or unintended consequences’ of the TEF proposal.  This is largely related to HEI behaviours and how they may respond in order to achieve higher TEF levels.  For example, to improve retention an HEI may make courses easier; or may change student recruitment profiles (i.e. accept less marginalised groups).  Another case in point is that those HEIs who do not score well will be prevented from improving simply because they have been financially restricted in raising fees.  Internationally, the TEF could also have an ‘adverse impact on UK’s international brand and the ability to recruit students, in the absence of similar information being published by our competitors’, the report notes.  Although countries will be ‘watching closely’, the UK will have to learn from its own mistakes and deal with those consequences over time.

The metrics for teaching quality is by no means an easy issue, and the Committee recognises this.  However, it does note that ‘Universities have been diligent in identifying reasonable concerns with the impact of any chosen metric, but…were less likely to propose viable alternatives.’ The Committee urges ‘universities to engage constructively on the technical consultation to ensure that the metrics chosen for the TEF are as robust and reliable as possible…’

The Committee also notes that ‘students are entitled to expect high quality teaching, given the costs they are incurring [and that] the Government should be sensitive to the perception that it is students who are having to fund an increase in teaching quality.’

A further confidence of both students and universities ‘will better enable students and taxpayers to hold universities to account for the service they provide. A multi-tiered TEF should only be introduced once Government can demonstrate its metrics have the confidence of students and universities. Achieving the necessary degree of confidence is in part dependent on the timetable for full introduction.’